From PoC to Production: Scaling Bright’s Training Simulations with Tribe AI & AWS Bedrock

To learn about the initial PoC, read this case study.

Introduction

Bright is a learning services platform that empowers companies like Nike and Aetna to train their teams through immersive, scenario-based simulations. These simulations are essential for preparing customer support agents to handle complex situations, such as managing order discrepancies or navigating insurance approvals. For instance, Nike's customer support agents use Bright's platform to engage in simulated conversations with virtual customers, addressing issues like incorrect shoe sizes or missing items in bulk orders. 

The platform allows agents to practice accessing internal systems like Salesforce or order management systems, identify root causes, and follow proper procedures. Moreover, it trains agents to handle a variety of customer personalities, from friendly inquirers to irate customers, ensuring they respond empathetically and in alignment with the brand's voice and customer-facing ethos. 

The Problem

The existing system at Bright required Learning and Development Managers at companies like Nike and Aetna to manually create detailed, if-else decision trees for various customer personalities and scenarios. This process was used to simulate a wide range of customer interactions, from angry customers inquiring about order updates to calm individuals seeking insurance pre-authorization. However, this manual approach was extremely labor-intensive and often fell short of covering the full spectrum of possible interactions, limiting the effectiveness and adaptability of the training programs. 

For instance, a manager might need to create scenarios for: 

An angry customer asking about an order update 

A happy customer inquiring about an order status 

An irritated customer trying to complete an insurance pre-authorization 

A confused customer lacking proper insurance or doctor information for pre-authorization 

The complexity of creating these simulations with extensive if-then-else branching was not only time consuming but also challenging. Managers had to provide verbatim responses for each potential interaction, which proved to be an inefficient method of preparing customer service representatives for the diversity of real-world scenarios they might encounter.

Challenge 

The core challenge lay in the inefficiency and limitations of manually constructing these conversation simulations. Learning and Development Managers needed to define every potential interaction, which involved crafting specific responses for various customer moods and scenarios. This approach required managers to anticipate and script out numerous possible conversation paths, considering factors such as: 

Customer emotions (e.g., happy, angry, confused, calm) 

Types of inquiries (e.g., order updates, insurance pre-authorizations) 

Customer demographics (e.g., age, geographical location) 

Communication styles and preferences 

This method was not only time-consuming but also failed to encompass the true diversity of real-world customer interactions. It was nearly impossible to create an exhaustive set of conditions to tackle the various responses that a learner might encounter in actual customer service situations. 

Solution 

To address these challenges, Bright developed a next-generation platform that revolutionizes the creation of customer personalities and interaction scenarios. The new system automates the process, making it significantly more efficient and comprehensive. Here's how the solution works: 

Natural Language Profile Creation: Instead of manual scripting, Learning and Development Managers can now describe customer profiles in natural language. They can specify traits such as: Age 

• Personality style 
• Geographical location 
• Communication preferences 
• Specific scenario details (e.g., seeking a prescription refill) 

Dynamic Interaction Simulation: These profiles are used to generate sophisticated bots that simulate these customer profiles. For example, it can create "Eva," an elderly customer seeking a prescription refill from Aetna. 

Comprehensive Scenario Coverage: The system allows Learning and Development Managers to focus on key scenario fork points in the conversation tree without needing to cover every single detail. Generative AI and Large Language Models (LLMs) augment and fill in the gaps of the conversation tree, providing comprehensive coverage that managers might not have time to create

manually. This approach ensures a more thorough and diverse range of training scenarios while significantly reducing the time and effort required from managers. 

Integrated Evaluation Engine: Tribe also developed a rating and evaluation engine as part of the solution. This engine assesses the trainee's performance across various metrics, like:

• Empathy levels 
• Adherence to company policies 
• Following the optimal path along the conversation tree 
• Ability to handle varying personalities for the same scenario 

This comprehensive solution enables companies like Nike and Aetna to create more realistic, diverse, and effective training simulations for their customer-facing teams, significantly improving their ability to handle a wide range of real-world customer interactions. 

AI / LLM Challenge addressed in the solution 

While leveraging LLMs to fill in gaps and handle unspecified paths in the decision tree greatly enhanced the system's flexibility, it also introduced new challenges that Tribe AI had to address: 

1. Consistency and Comparability: The use of LLMs introduced variability in bot responses, which posed a challenge for fair evaluation of trainees. It was crucial for companies like Nike or Aetna to be able to compare the performance of multiple employees who underwent training, rating who did well and who didn't. This required a certain form of "apples to apples" comparison between the paths taken, the responses provided by bots like Eva, and how human trainees responded to those responses. Tribe AI developed mechanisms to maintain a delicate balance between variability and consistency in bot responses, allowing for meaningful comparisons between trainees' performances while still providing diverse, realistic interactions.

2. Customization and Control: After numerous training sessions with a bot like Eva, Learning and Development Managers often gathered feedback about the bot's quirks and behaviors. This necessitated a way to fine-tune bot personalities without breaking critical conversation points or paths. Tribe AI engineered a system that allows for this level of steerability and control over the LLM, ensuring that critical conversation points remain intact while allowing for adjustments to enhance the training experience. This was achieved without making it too burdensome for learning managers to specify everything in great detail.

3. Integration of Real-world Data: Companies like Aetna and Nike often wanted to incorporate their own past data from real-world customer interactions to create more authentic bot personalities. To address this need, Tribe AI designed the system to incorporate actual customer interaction data, allowing for the creation of bot personalities based on real-world customer profiles and behaviors. This feature significantly enhanced the realism and relevance of the training scenarios.

4. Latency and Real-time Interaction: A major challenge with a purely GenAI system is latency, especially in scenarios where conversations between trainees and bots could last for hours. In such cases, the GenAI system needs to keep track of the conversation's progress, covered and uncovered topics, and generate responses in real-time. To address this complexity, Tribe AI implemented advanced indexing and smart retrieval systems. These innovations enable the GenAI system to quickly access relevant information and generate timely responses, even supporting voice-to-voice conversations between trainees and bots like Eva. 

Implementation 

The implementation leveraged AWS services and Claude 3.5 Sonnet from Anthropic via AWS Bedrock. The primary component of the solution was the Bot Interaction Engine. 

Bot Interaction System 

This system facilitates real-time online processing. It employs Python and FastAPI, along with AWS ECS Fargate, to respond to trainee interactions in real-time. The system's complexity necessitated the development of custom indexing and AI-powered retrieval engines (backed by AWS RDS and pgvector) by Tribe AI engineers to support the use case. The diagram below illustrates how the bot interaction system functions within the FastAPI service, which is run on AWS ECS Fargate and sits inside a VPC. 

‍

Architecture 

Architecture Details 

The Bright Learning Platform utilizes a modern cloud-native architecture built on AWS, consisting of four main components: 

Frontend Layer 

• This is a AWS Amplify hosted React web application. This is part of their existing platform. No changes / development was done here as part of the Tribe AI engagement.

API Gateway Layer 

• Leveraged Bright's existing AWS API Gateway infrastructure, adding new routes specifically for the conversation bot functionality 
• The existing gateway continues to handle request routing, composition, and protocol translation 
• Utilizes the platform's established security, monitoring, and rate limiting capabilities 

AI Processing Engine 

• Core AI processing logic implemented as a Python FastAPI application 
• Deployed on Bright's existing AWS ECS Fargate cluster within their VPC, utilizing their established auto-scaling capabilities 
• Leverages Bright's existing Amazon RDS infrastructure for storing AI chat session state and management data alongside their core application data 
• Establishes connection to Anthropic's Claude 3.5 Sonnet model via AWS Bedrock for real-time conversation processing and response generation

The entire system is contained within AWS Cloud, with the application components running in a dedicated VPC for security. This architecture ensures scalability, reliability, and secure handling of training simulations while maintaining low latency for real-time interactions between trainees and AI powered conversation bots.