Creating Personalized Experiences & Increasing Online Sales

An Interview with Craig Kistler from Signet Jewelers

Meet Craig Kistler, the Senior Director of Personalization & Experimentation at Signet Jewlers. He is responsible for their seven key brands: Kay, Kay Outlet, Jared, Zales, Zales Outlet, Banter (formerly Piercing Pagoda), and Peoples Jewelers. He is an award-winning CRO and jewelry industry leader.

Given that Craig is someone paving the way for CRO in the jewelry industry -- as the industry as a whole is digitally lagging behind -- I wanted to have an in-depth interview with him to better understand what he's doing. Spoiler, I'd say he's actually running one of the most mature optimization programs I've heard of over the years. 

Tell us your professional story. Where did you start? How did you get to where you are now?

I went to college for graphic design and started my career web design in the late '90s. I transitioned into user experience, which became a true passion. This passion led me to my first big break in UX—though not without a little improvisation.

When I applied for a UX role at WhittmanHart, I claimed to have experience in creating wireframes and site maps. The truth? I’d never made one in my life. During the second round of interviews, I was asked to show my portfolio. With just a few days to prepare, I dived into Photoshop (the only tool I knew at the time) and created a set of wireframes and a site map as if I were designing a website. Somehow, it worked, and I got the job. This experience taught me the importance of adaptability and the power of diving headfirst into new challenges.

From there, I moved on to Progressive Insurance, where I had the privilege of using their in-house usability testing lab. This hands-on experience cemented my belief in the importance of design decisions, especially when you see the real-time impact they have on users. Leading the UX strategy for Progressive’s first online claim reporting process was a highlight of my time there.

At American Greetings, I got my first real taste of eCommerce, understanding customer needs and expectations when they’re trying to find that perfect card. This was crucial in shaping my approach to user-centered design and optimizing the online shopping experience.

When I joined Signet Jewelers, I was tasked with building their UX practice from the ground up. Over the course of five years, I developed and implemented UX strategies that enhanced the shopping experience across our brands. My work in UX laid the foundation for everything that followed, including my transition into A/B testing. It was during this time that I realized the power of data-driven design to continuously improve user experiences.

Now, as the Senior Director for Personalization and Experimentation at Signet, I oversee the personalization efforts and experimentation strategies across all our brands. My role involves blending creativity with data, ensuring that every decision we make is informed by real user behavior. The result? A personalization strategy that increased conversion rates and revenue adding over $45 million in incremental revenue in 2023. I’ve also been recognized with industry awards, including the 2022 Personalization Thought Leadership Winner and the 2018 Award for Best Testing and Optimization.

Explain your current role in more detail. What brands are you responsible for?

As the Senior Director for Personalization and Experimentation, I oversee the onsite personalization efforts and experimentation strategies across seven key brands: Kay, Kay Outlet, Jared, Zales, Zales Outlet, Banter (formerly Piercing Pagoda), and Peoples Jewelers. My role is central to shaping and executing our personalization strategies across these diverse ecommerce sites.

In this capacity, my team and I are responsible for running all onsite personalization campaigns and conducting rigorous A/B tests to ensure that our strategies are both data-driven and impactful. We work closely with product and marketing teams to design, implement, and refine campaigns that enhance the user experience and drive conversion rates. By leveraging insights from user behavior and continuous experimentation, we can optimize the customer journey and deliver personalized experiences that resonate with our audience.

Can you elaborate on Signet’s CRO program at all? e.g., testing velocity, research efforts, your personalization efforts, etc.

Our program focuses on two key areas: personalization and experimentation. While there's some overlap, each serves distinct purposes. Experimentation is about partnering with product and marketing teams to answer critical questions, such as whether we should invest time in building a new feature or how we can enhance a specific marketing experience. We also use experimentation to validate user research, ensuring what users say in a lab translates into real-world behavior on our site. For instance, with millions of monthly visitors, we can test if lab findings hold up when visitors shop from home.

We start our personalization strategy with macro audiences because understanding broad segments allows us to make impactful changes that resonate with large groups of visitors. For instance, we begin by identifying how visitors shop—distinguishing between those looking for engagement rings versus those interested in other products. These macro audiences are essential as they provide the foundation for a more targeted, relevant experience.

Starting broad is critical. It ensures that our personalization efforts are not only scalable but also meaningful. By focusing on significant segments, like those with varying purchase intents (low, medium, high) or product intents (engagement rings vs. fashion items), we can make substantial improvements that affect a large portion of our audience. From these macro segments, we further refine our approach, tailoring experiences to more specific needs, such as deal seekers versus luxury shoppers.

Once we’ve identified these key audiences, we then experiment with finding the right message, creative and placement. We test hypotheses to validate the effectiveness of these segments, ensuring that the personalization we implement truly enhances the user experience. This iterative approach allows us to refine our strategies continuously.

A quick note on personalization: While the industry often talks about one-to-one personalization, my approach is more about crafting relevant experiences for groups of like-minded visitors. For example, someone shopping for an engagement ring has different needs than someone buying a birthday gift or shopping for themselves. We use what we know to create better experiences for each scenario without needing to rely on granular individual data. It’s about making the experience more relevant based on broader behavioral patterns.

Although we’re a relatively small team, we run a high volume of campaigns—around 40-50 per quarter—and we aim to double that in the next six months. Historically, our focus was 80% on experimentation and 20% on personalization. However, this has since flipped, with personalization now taking the lead.

How mature do you think the jewelry industry is right now relative to digital efforts?

The jewelry industry has traditionally been seen as one that requires in-person experiences—much like car buying—where people feel the need to see, touch, and try before they buy. For a long time, this meant that digital efforts lagged behind. However, the pandemic was a major turning point. Businesses that had been slow to embrace digital were forced to adapt or risk being left behind. At the same time, a surge of new startups has been shaking up the industry, pushing everyone to step up their digital game. The need to innovate and compete in this new landscape is driving significant growth in digital strategies across the board.

Brands that will thrive are the ones that understand it's not just about being digital or in-store—it’s about being relevant whenever and wherever the customer wants you to be. It’s also crucial to focus on digital strategies that actually benefit your customers. Too often, brands fall into the trap of following trends just because “brand X is doing it,” which is the digital equivalent of keeping up with the Joneses. Success lies in carefully evaluating what truly adds value to your customer’s experience, rather than simply jumping on the latest bandwagon.

How mature do you think the jewelry industry is right now relative to CRO efforts? (If it’s immature, why do you think that’s the case?)

I’d say that truly successful long-term optimization programs in the jewelry industry—and frankly, across many industries—are still pretty immature. While improving conversion rates isn’t a new concept, too often the focus is on boosting a single metric without considering the broader impact on the overall customer experience.

CRO is about more than just tweaking a number; it’s about enhancing the entire customer journey. Unfortunately, many companies don’t dedicate the necessary time and resources to grow and nurture their optimization efforts. They might hire an agency to run a few tests based on “best practices,” which often aren’t tailored to their specific customers. When the results are mediocre, they dismiss CRO as ineffective.

Additionally, CRO is sometimes treated as a side project or a “hobby” rather than a core business function. Without a dedicated team and a long-term commitment, it’s hard to build a program that drives sustained, meaningful improvements.

What would you say to other jewelry stores, independents or corporate chains, that either aren’t yet aware of digital’s potential or aren’t fully bought into digital efforts? 

I would say to any jewelry store—or any retailer for that matter—digital is only going to become more sophisticated. Even customers who plan to make their final purchase in-store almost always start their journey online. It’s not about having all the bells and whistles that a large retailer might have, but about understanding where your customers are and meeting them there.

Digital doesn’t have to be overwhelming, but it needs to be intentional. Consider why customers shop with you—whether it’s for your unique products, exceptional service, or something else—and think about how you can enhance that experience. This could mean better storytelling around your products, making them easier to find, offering online consultations, or creating a seamless blend of online and in-store interactions.

The key is to focus on how you can make the shopping experience better for your customers, whether they’re engaging with you online, in-store, or both. This understanding will be critical for sustainable growth. As digital continues to evolve, those who adapt will be the ones who thrive.

Signet Jewelers seems to be extremely mature relative to digital efforts, particularly CRO. Would you agree or disagree?

If yes, are you able to elaborate on why you think Signet is so mature? Is it related to the business’ culture? Leadership?

I’d say Signet is maturing—think late high school, early college. We’ve definitely made big strides, especially during the pandemic, which pushed us to accelerate our digital efforts. From an optimization standpoint, we’ve been ahead of the curve, leaning into CRO for the last 6.5 to 7 years. But we’re still pushing boundaries, and in that sense, I always want us to keep maturing.

Our maturity comes down to a few key things. Leadership has been supportive from day one, always viewing digital as a huge opportunity. They’ve brought in some talented leaders and product owners, which has made a big difference. Over the years, I’ve seen a significant shift in how teams think and act, all driven by a better understanding of what our customers want.

Recently, our leadership team asked us to accelerate initiatives to close out a quarter strong. In response, we doubled the number of planned campaigns, which led to a significant bump in incremental revenue. This effort not only boosted our results for the quarter but also served as a catalyst, helping our senior leadership team fully grasp the power and potential of crafting onsite behavioral personalizations.

That said, we’re a big company, and sometimes old ideas resurface and slow things down. Culture plays a big role here, and getting the new and old to work together is how we’ll keep moving forward.

What are the biggest learnings you’ve had in the recent past relative to CRO? What has made Signet so successful with CRO?

One of the biggest learnings we've had is the power of integrating experimentation with personalization. While we've shifted more focus toward personalization, experimentation remains at the core of everything we do. We continuously test and refine our strategies and messaging within each segment to ensure we're delivering the best possible experience for our customers.

For example, we’re not just curating personalized assortments and making products easily findable; we’re doing so for different audiences at the same time. This means that, on any given page, we might showcase a luxury collection to one group while highlighting a value-based assortment to another—all driven by ongoing experimentation.

We’ve also learned that conversational approaches, where we recognize and respond to visitor needs in real time, significantly impact interactions. Tailoring experiences based on purchase and product intent, and continuously testing these approaches, has been crucial to our success.

These learnings have been integral to our progress, allowing us to continually refine our strategies and deliver meaningful, data-driven experiences.

What are the biggest learnings you’ve had in the recent past relative to digital efforts overall for the jewelry industry specifically?

One of the biggest things I’ve learned is that customers will always find a way to surprise you. We’ve seen plenty of cases where qualitative research pointed us in one direction, only to have experimentation prove us wrong. This happens a lot with so-called "best practices." Just because something works elsewhere doesn’t mean it’ll work for everyone. I’ve seen “tried and true” strategies—like “complete the look” or social proof—just not resonate with our visitors.

That doesn’t mean these ideas are bad, but they need to align with what your visitors are actually trying to accomplish and how they are thinking. Best practices are a good starting point, but they shouldn’t be taken as gospel. They need to be tested in the context of your visitors and, more importantly, your specific audiences. There are some core principles—like solid UX and usability—that are reliable, but it’s crucial to stay skeptical of any “expert” advice until you’ve seen it work with your own customers.

What do you think the biggest areas of opportunity are that you’re excited to tap into relative to CRO specifically and/or digital efforts at large?

Anything aspirational that you want to do but might not be able to for whatever reason, too?

One of the most exciting opportunities we’re exploring is understanding customer intent at a much more granular level. We’re identifying specific, measurable signals that indicate whether a visitor is likely to make a purchase or if they’re about to exit without converting. For example, by analyzing patterns in browsing behavior, time spent on product pages, or how a visitor interacts with content, we can gain insights into where they are in their journey.

The real power comes when we start stacking these intents together. By combining different signals, we can build a comprehensive, real-time picture of what a visitor is thinking and what they might do next. This allows us to tailor the experience even further, offering more relevant navigation, content, or products at the exact moment they’re most likely to make a difference.

We’re also pushing the envelope with AI-driven personalization, but we’re being very thoughtful about how and why we use it. While AI is a big buzzword right now, we’re not interested in using it just to say we’re on the bandwagon. For us, it has to be integrated in a way that feels authentic and genuinely enhances the customer experience. We’re also aware that AI is still new to many people, and not everyone trusts it, so we’re being cautious in how we use it to ensure it doesn’t negatively impact any of our visitors.

On the aspirational side, I’m excited about the potential for cross-channel personalization. Imagine a customer starts their journey online and then walks into a store where the experience seamlessly continues based on their online behavior. The complexity of pulling this off—getting all parts of the business perfectly aligned and ensuring the technology supports this fluid experience—is a big challenge, but the potential impact on customer satisfaction and loyalty is huge.

What are the biggest challenges you’ve seen for Signet that would relate to the jewelry industry as a whole?

One of the biggest challenges I’ve observed is the ambiguity around terms like CRO and personalization. These terms mean different things to different people, leading to misalignment. For instance, I see CRO as optimization across the board, not just conversion rate optimization. However, when the focus is solely on that one metric, it can narrow the perspective. This narrow focus often leads teams to optimize for short-term gains without considering the broader impact on the overall customer experience.

The same goes for personalization. With all the talk about the death of cookies and the push for zero- and first-party data, large customer data platforms are pushing their tools as the only way to achieve personalization. But because these companies are big players, their messaging often gets accepted without question. The challenge is that the return on these investments doesn’t always meet expectations. There’s definitely a place for a CDP, but brands can achieve a lot before diving into that level of investment.

My team used to face these challenges more frequently, but we’ve made a concerted effort to demonstrate the benefits of each method in terms of incremental dollars. Since we run all personalizations as experiments first, we can measure the impact using specific data types. Knowing that the level of effort to build each campaign is the same, showing ROI on something that reaches millions of visitors compared to something that reaches thousands has proven effective. That’s not to say we haven’t been effective using data from a CDP—we have—but right now, the opportunity is bigger elsewhere.

Another challenge is internal understanding and the need for control over messaging. Many brands still rely on “time-tested” methods that have worked in the past, making it difficult to embrace newer strategies that might feel unproven. This reliance on old playbooks can hinder progress, as getting teams to trust data over their gut instincts is an ongoing challenge.

For those brands that don’t have enough data to test, what would you tell them to do (or not do) for CRO and digital efforts?

Even if you’re short on traffic, there’s still plenty you can do to improve your CRO efforts. Start by digging into the data you do have. Where are people entering your site? Where are they dropping off? Which channels are driving traffic, and what’s that traffic doing? For example, if most of your traffic comes from social media but bounces quickly, that’s something your data will flag.

Beyond basic analytics, tools like session replay (think Hotjar) can be incredibly valuable. Watching how real users navigate your site will reveal where they hit snags and what’s working well. Sometimes, seeing these interactions firsthand can uncover issues that raw data alone won’t show.

Usability testing is another powerful tool in your arsenal. With platforms like UserTesting, you can observe users in action, understand their pain points, and identify what needs fixing. Often, addressing these pain points can lead to noticeable improvements in conversion rates, even with limited data.  

For example, I’ve run tests where we ask visitors to imagine they’re buying a gift for someone special, with a specific budget in mind. By starting them on a common page, like a PLP (Product Listing Page), and observing their unprompted actions—whether they scroll, filter, or interact with important elements—we gather valuable data on what’s working and what isn’t.

It’s about making the most of the insights you have and using them to drive smart, impactful changes.

What is your primary CRO tool stack?

We use Dynamic Yield for our campaigns. Our audience rules and tracking is built inside of DY. While we have Adobe CDP feeding data into DY, we don’t rely on it as the core of our program.

How do you manage your personalization efforts from a project management perspective?

It's an evolving process. Especially with the more campaigns we run. 

We use a combination of strategies. First, we focus on macro audiences, which are less than a dozen. Second, we use personalization concepts and patterns we've proven to be effective which allows us to extend them to other sites easily.

While not sexy, we currently rely on spreadsheets with some automation triggers. Once a personalization proves successful, it’s added to the spreadsheet, which tracks the site, page, audience, and launch date. This allows us to pull to track what we have running and then alerts us to check or re-run the experiment to ensure continued effectiveness.

Additionally, we maintain a library of previously run experiments that we share across the team, ensuring that everyone can leverage past insights.

Anything else you want to share? 

As digital keeps evolving, I’m focused on pushing the boundaries with personalization and experimentation. I’m excited about how we can further integrate AI in a way that genuinely enhances the customer experience, and I’m always looking for new ways to create a seamless blend between online and in-store interactions.

None of this happens without the dedication and collaboration of my team. We’ve built an environment where testing, learning, and refining are part of our DNA. The challenges are always there, but so are the opportunities to innovate and make a real impact.

Looking ahead, I’m focused on making sure we stay ahead of the curve and set the standards in this space. There’s a lot more we want to achieve, and I’m excited about what’s next.