AI Training Specialist

at UnifiedBeez Ltd

About the Role

As an AI Training Specialist at UnifiedBeez Ltd, I lead the design, implementation, and maintenance of the company's intelligent conversation-tagging platform. UnifiedBeez is building smart, scalable conversation intelligence solutions for businesses across Africa and beyond.

Core Responsibilities

• Conversation Tagging System: End-to-end ownership of UnifiedBeez's multi-layer tagging system using Rules → Classifier → LLM fallback architecture, from design to production operation.
• Rules Layer: Design and maintain human-editable YAML/JSON rules files with keywords, regex patterns, and negative phrases. Implement versioning, change reviews, and comprehensive audit trails.
• ML Classifier Development: Build and maintain lightweight multi-label classifiers using sentence embeddings with One-vs-Rest logistic regression/SVM. Set per-tag thresholds and calibrate probabilities for optimal performance.
• LLM Fallback Implementation: Implement cost-controlled LLM fallback systems using GPT-4o-mini and Gemini with strict JSON schema/function calling, caching, batching, and confidence-based escalation.
• Data Engineering & Active Learning: Bootstrap datasets with weak labels from rules, run active learning loops, curate gold standard datasets, and manage dataset versioning for reproducible training and evaluation.
• API Development & Integration: Design, expose, and maintain the /tag endpoint and related services. Ensure backward compatibility, performance SLAs, and seamless integration with backend and frontend teams.
• Automation Engine: Define tag-triggered automations with event schemas, implement webhooks/queues with retries and backoff, map tags to workflows (assign, escalate, notify, autorespond, CRM updates), and monitor automation success rates.
• Monitoring & Metrics: Build comprehensive dashboards for coverage, per-tag precision/recall/F1, LLM usage percentage, latency, and cost per message. Ensure every applied tag logs a complete audit trail (rule ID, model version, confidence, LLM trace).
• Performance Optimization: Optimize inference latency and throughput, implement load testing, rate limiting, graceful degradation paths, and plan model version rollouts using blue/green and canary deployment strategies.
• Security & Compliance: Enforce access controls, implement encryption for data in transit and at rest, apply data minimization and redaction techniques, and adhere to company security policies and applicable laws.
• Documentation & Collaboration: Maintain clear documentation for rules, models, endpoints, automations, and operational runbooks. Produce weekly summaries of progress, blockers, and priorities. Work closely with Product, Backend, Frontend, and PM teams.

Key Technologies & Tools

• Machine Learning: Sentence Embeddings, One-vs-Rest Classification, SVM, Logistic Regression, Model Calibration
• LLMs: GPT-4o-mini, Gemini, Function Calling, JSON Schema Validation
• Backend: Python, RESTful APIs, Webhooks, Queue Systems
• Data Engineering: YAML/JSON processing, Active Learning, Dataset Versioning
• DevOps: Blue/Green Deployment, Canary Releases, Load Testing, Performance Monitoring
• Cloud Services: AWS, Caching Systems, Database Management
• Security: PII Redaction, Data Encryption, Access Control, Compliance

Key Performance Metrics

The role involves meeting stringent performance targets including:

• Precision ≥ 85% and Recall ≥ 70% for top revenue-relevant tags
• ≥ 75% coverage through Rules + Classifier without LLM usage
• LLM usage rate ≤ 10% (rolling 30-day average)
• Latency p95 ≤ 120ms per request at 250 rps
• Cost per message ≤ $0.0010 including cache hits
• Cache hit rate ≥ 75% for LLM calls
• 99.9% monthly uptime for tagging services

Impact & Value

This role is critical to UnifiedBeez's mission of providing intelligent conversation intelligence to businesses. By developing a robust, scalable, and cost-effective tagging system, I'm helping organizations automatically categorize and understand customer conversations, enabling automated workflows, better insights, and improved customer experiences across various industries in Africa and beyond.