Label in the Loop
An efficient system for effective training of machine learning models through a labeling process.
Label in the Loop is an enabling technology for data driven organizations that need collaboration with or supervision of domain experts over an unlabeled body of data.
LITL’s core value proposition is smart selection of data samples and smart designation of those samples to domain experts.
- The cold-start mitigation methods in LITL (selecting the initial data batch) achieve a stable result and the quality of the 80th percentile (or higher) of the quality distribution of the (unstable) baseline method.
- The method of extracting expert knowledge reaches over 110% of the quality of your best expert.
- On 10% of the data, LITL achieves over 80% of the quality of the model trained on 100% of the data (better models faster and cheaper).
LITL will help you with:
01.
Reduction of time and cost of machine learning model training
02.
Training machine learning models in no data or small data scenarios – allowing for gradual increase of model quality and its adaptation to changing data or requirements
03.
Experts analytics (i.e. experts assignment, experts consensus and experts performance estimation)
How does it work?
LITL is an active learning system to effectively train ML models in the setup of unlabeled data. Specifically, LITL selects samples from an unlabeled pool that will have the most influence on the model’s performance and assigns them to experts according to their estimated performance on similar samples. Experts’ assignment process itself balances between exploration and exploitation of experts’ knowledge to estimate their performance more effectively. The whole process works iteratively by retraining the model with consecutive batches of samples. As the model learns on an increasing labeled dataset, it changes its uncertainty towards unlabeled samples, on which the sampling process is based.
Features:
Active learning sampling
Initial batch selection
Experts assignment
Experts consensus
Experts performance estimation
New classes identification
Active learning sampling
Initial batch selection
Experts assignment
Features:
