April 07, 2022 | Ujjval Patel, Director of Solutions and Consulting, Hi Marley

Taking an InsurTech Solution for a Test Drive Part II: Implementing a Successful Pilot

As discussed in Part I, the first step in choosing the right InsurTech solution is putting it through its paces in a structured, low-risk environment. Once you identify the pilot’s scope, approach, objectives and measures of success, it’s time to get down to the nitty-gritty. This article will provide a detailed roadmap to effectively “test drive” an InsurTech product.

Create the Right Pilot Team

Designing a thoughtful pilot that includes a variety of users will greatly assist in the post-pilot assessment. It’s critical to build a strong cross-organizational pilot team that includes a mix of tenure, experience, technical understanding and willingness to take risks—everyone will bring a valuable perspective. You’ll want to hear their concerns early in the process, not when launching the product.

On the carrier side, the most successful pilots typically include:

  • Executive Sponsor/Champion
  • Pilot Project Leader
  • Pilot User Group
  • Systems/Analytics Lead
  • Other discipline-specific SMEs as needed

To ensure the pilot runs smoothly, you must also have a strong support team from the InsurTech partner. On the solution provider side, an optimal team often includes:

  • Relationship Manager
  • Pilot Leader
  • Customer Success Manager
  • Technical Implementation Manager
  • Business SME(s)

Having the correct people in the right roles facilitating the pilot will accurately measure whether the solution is the right fit.

Plan for Pilot Deployment Challenges

Congratulations! Your organization has now started the experiment. But what common challenges should you look out for?

Sometimes, organizations try to change the scope of the pilot mid-flight. Certain changes may be warranted if they create valuable insights or remedies to specific constraints, such as too much or too little claim volume, employees out sick, unexpected business interruptions, catastrophic events, etc. However, to prevent the pilot from spinning out of control, before making any changes in scope, the pilot team needs to ask some questions to assess if the pilot should continue:

  • What insight are we trying to derive from the shift in pilot direction?
  • Instead of changing direction, is there an alternative way to gather that insight?
  • How much disruption or risk will the change cause?
  • Can we do a subsequent pilot to answer other specific questions?

The goal isn’t to add rigidity to the pilot process; it’s to make sure you can isolate the signal from the noise and avoid unnecessary changes that might derail the project. If you decide to make changes, make it part of your practice to document them and note how they may impact the product implementation.

Make Post-Pilot Learning a Priority

Due to your excellent pilot planning and implementation, you were able to generate great objectives, measures of success and accurate benchmarks. Now, it’s time to take the final measurements and determine the solution is a fit for your organization.


Let’s say there is a significant difference between the types of claims that came in during the pilot compared to your overall annual claims makeup. In this situation, weight the pilot results to reflect the overall annual portfolio given an adequate sample size per claim type.


With the pilot’s results in hand, you can extrapolate and estimate implementation costs vs. the return on investment. Hi Marley can help in this process, bringing our experience and industry knowledge to the table.

Plymouth Rock, for example, handled 2,500 claims during its Hi Marley Pilot and experienced increased policyholder satisfaction ratings, reduced rental days and lower operational costs. Based on the successful results in claims, Plymouth Rock expanded Hi Marley across all customer and agent-facing interactions.

Customer Satisfaction

Impact on customer satisfaction is a critical metric to evaluate at the end of a pilot, and Hi Marley provides industry benchmarks where available. In addition to the overall satisfaction score, carriers may want to explore things like how one point of extra Customer Satisfaction Score (CSAT) equates to retention improvement? Or, how will Net Promoter Score (NPS) improvement impact top-line growth? While these insights may require collaboration amongst the underwriting, marketing, sales and analytics departments, any indication of improved customer satisfaction should help when making a final decision.


Increased efficiencies are something carriers should look at in the post-pilot evaluation. Operational efficiencies impact overall costs, customer satisfaction and productivity. For example, when Florida-based Bankers Insurance experienced compelling results using Hi Marley for claims, it worked closely with us to pilot the platform in underwriting to streamline the property inspection process via text messaging. Throughout the pilot, Hi Marley leveraged its insurance expertise to help Bankers’ agents, underwriters and policyholders to refine the experience and platform in a range of underwriting use cases. As a result of the pilot, Bankers experienced dramatic improvements in its inspection timeliness. Completed perimeter inspections increased from 46 percent to 88 percent, and four-point inspections went from 33 percent to 83 percent.

User Feedback

Integrating insights from frontline employees who participated in the pilot is critical to the InsurTech solution’s success. Too many organizations make a “top-down” decision only to find out later that the solution doesn’t solve the problems for which it was intended. The most invested folks need to be heard loud and clear.

Be sure to capture two levels of insight. First, conduct individual interviews to capture qualitative feedback. Then, using post-pilot user surveys, gather quantitative findings.

Ohio Mutual Insurance Group, for example, piloted several solutions over the years, but the solutions continued to fall flat. Rather than settling for a “good enough” option, they continued to search for a product that met their requirements. They discovered Hi Marley and a cross-functional ran a pilot. As a result, claim handlers experienced 100 fewer calls per person per month. With less time on the phone, they could move claims along faster. In addition, insured customers appreciated the ease of text messaging.

Ideally, if capacity allows, it’s also a good idea for pilot teams to have observers watch users in action or, at a minimum, go back and review specific test cases to spot potential challenges.

One of the largest property and casualty insurance providers in the US ran a Marley Mobile pilot to explore the convenience of the mobile app for field adjusters to get in touch with policyholders via text message. Field adjusters quickly experienced less time wasted looking for a Wi-Fi connection or somewhere to pull over their car so they could open their laptop and communicate on the case. It simplified the workflow, and they had an improvement in response time. In about six weeks, the organization’s mobile time to first contact improved by 80 percent.

Hi Marley can assist in all of these steps, so you can gather this feedback without overburdening your teams. Ultimately, the goal of the post-pilot activity is to come to a consensus around a go/no-go decision. If the decision is “go,” it’s important to uncover latent feature requirements and potential issues that, if not addressed, could represent significant blocks to success. Most potential issues can be resolved with low-impact workarounds. Our goal is to be your partner throughout this journey.

Make Insurance Lovable

Before considering a production implementation of any InsurTech solution, we strongly encourage carriers to run a pilot. Structuring a sound experiment allows potential production users to see the product in action and the organization to clearly quantify the solution’s ROI.

At Hi Marley, we know the value our product creates for clients, and that’s why we encourage, support and facilitate this process, offering it to prospective clients at no cost. So, let’s go for a test drive. We know it will make insurance lovable.

Next Posts

Taking an InsurTech Solution For a Test Drive Part I: Designing a Successful Pilot

View Now

Hi Marley, the Zipwhip Replacement for Insurance

View Now

Improving Claims Best Practices with Insights from Conversational Text Data

View Now
Back to Blog