Twitter has announced the release of their new RA-HF (Recommended Algorithm High Fidelity) Transparency Report, which includes the services and handlers used in the construction of a tape of recommended messages shown by the user on the main page (Home Timeline). The report aims to provide transparency and the possibility of an independent audit of the algorithms used, and Twitter has expressed their willingness to accept feedback from the algorithm that can improve their operation.
The code for the RA-HF algorithm has been made open source under the AGPLV3 license, with implementation using SCALA programming languages (53.8%), Java (29.7%), Starlark (6.3%), Python (4.7%), C++ (2.4%), and Rust (1.5%). Twitter has posted a separate repository related to the machine learning models they use, with components for the formation of advertising recommendations not being published.
Building a tape of recommendations involves three main stages, according to Twitter. The first stage involves extracting the best tweets from different sources, including their indexing system, which covers messages of people who have a subscription (in-network), and a layer used to extract messages taking into account the interaction of the current user. The second stage involves the ranking of selected tweets using two systems, the Light-Ranker model, which uses the search index, and the Heavy-Ranker neural network.
Twitter’s new transparency report and open source release of their RA-HF algorithm demonstrates their commitment to transparency and continued improvement of their recommendation algorithm.