Eight Weeks Building Langflix's Shorts Recommendation System
I want to look back at the analysis and decisions we made over eight weeks of building the Shorts recommendation system.
From recommendations that keep people watching to recommendations that encourage learning
Most Shorts recommendation systems aim to keep people watching longer. Because Langflix is an English-learning app, we chose a different priority: encouraging users to learn. We gave learning actions more weight than watch time when designing and configuring the models.
Building the data foundation for recommendation
At first, I thought the most important part would be building a good model from the available data and using it to define recommendation logic. As the project progressed, I realized how much else matters beyond the model. Personalized recommendations need appropriate video embeddings. We revised the embedding process, generated new content vectors, and implemented a new clustering and storage workflow. We also calculated vocabulary difficulty against a CEFR dictionary when ingesting videos.
Gradual personalization based on user history
The model design also turned out differently than I first expected. We had planned to build one complete model, but the Two-Tower model we initially favored could make inaccurate recommendations for new users. We therefore added content-based filtering for users with little viewing history. This let us introduce personalization gradually. Once a user had enough history, the Two-Tower model could use a neural network to represent their preferences.
What I learned while building a recommendation system
The project taught me a great deal.
Operations matter as much as modeling
First, I learned the importance of operating a recommendation system, not just building its model. We spent as much time making it work in the existing production environment as we did configuring the models. New videos needed embeddings and clusters automatically. Trained models and item vectors had to be published to AWS DynamoDB. The video rankings used by content-based filtering needed periodic refreshes so every part of the system could use current data. We also designed fallbacks. Data generation, versioning, fallbacks, monitoring, and A/B testing all had to connect before this became a working recommendation system.
Shared standards make collaboration possible
Second, we needed a shared set of standards across the team. We began by defining our target metrics. After deciding what counted as a good response to a video view, we calculated model scores and metrics against that definition. It helped each person build their part of the system toward the same goal. The three models in the final system also used the same eligibleItems set. We first determined which Shorts a user could receive, then let each model rank those candidates. That way, the system would work whichever model was selected. Our shared starting point was the complete list of Shorts eligible for recommendation.
Limitations and next steps
There were also things we could not complete. We did not run online experiments long enough to verify each model's effects on learning actions and retention. Difficulty currently acts more like a safety guardrail; adapting it to each user will require further work. Still, improving Langflix's recommendations made me think deeply about how to move beyond encouraging views toward saving expressions, practicing speech, and naturally continuing to learn. I learned a lot from the project, and I am grateful to the Theta One team for their help and the opportunity to grow.
