The case for hybrid models (machine learning and actuarial)

Actuarial science and data science are different traditions, each with their own learning backgrounds, desired outcomes, standards for good work, and skillsets.

Actuaries tend to have mathematical modeling backgrounds that they learn through a combination of studying and work experience. This allows them to develop good understandings of the businesses they operate in, and they tend to be experts at connecting analytical methods to the business problems they or their employers face. Their mathematical backgrounds often place emphasis on a priori reasoning and of gaining an intuitive quantitative understanding of the world around them. While actuaries do work with and depend on data to a large degree, data scientists are even more focused on data.

Data scientists are generally less trained in math and more trained in statistics and computer science than actuaries. They have skills and tools that allow them to work with larger, more detailed datasets than actuaries typically can, and they can generally automate their work better. Because data scientists tend to have less training on the job and more training in the academic world, they can place more emphasis on statistical rigor than actuaries, but sometimes at the expense of business objectives.

Data scientists and actuaries are not interchangeable. For example, you wouldn’t hire an actuary to build a recommendation system for a streaming service and you wouldn’t hire a data scientist to sign off on a reserve statement for an insurer. However, there is an area where the functions of data scientist and actuary overlap: predictive modeling for financial security systems like insurance.

For this use case, in theory, actuaries and data scientists should be complementary. And some teams know this. However, I wish to take it a step further. People often think of machine learning predictive models and actuarial models as mutually exclusive. However, they each have different strengths and they should be combined in a way that combines the benefits of both of them.

Machine learning is great at learning detailed, sometimes nonlinear, patterns from large datasets. However, it’s clueless about anything outside of the data it gets to train on. On the other hand, actuaries have broader knowledge and they bake that knowledge into their models. For example, an actuarial model might incorporate information from three sources to make its predictions: a claims cost dataset, medical contracts and price lists, and some general industry trends.

In order to combine the depth of machine learning with the breadth of actuarial knowledge, we might want to use machine learning to build a model by analyzing the claims dataset and then combine that model with the other knowledge the actuary has. There is no formulaic way of doing this that always works perfectly. However, there are some options, and we will dive into them in a more detailed post.