PLATFORMS
Three closed platforms
One shared desktop, perception and review foundation
CREATOR PARTNERSHIPS · AI WORKFLOW AUTOMATION · COMPUTER VISION
Automating a mobile-only partnership workflow without an API.
We built a Windows application that observes rendered mobile screens, reconstructs creator profiles, evaluates campaign fit, prepares personalised outreach and carries out approved actions with visual verification.
- Client
- Confidential
- Sector
- Creator partnerships
- Engagement
- Custom internal application
- Status
- Delivered
INTERFACE
No API or DOM
Visual models, OCR, geometry and screen-state logic
CONTROL
Human-approved
Verification and no blind retry when ambiguous
PROJECT SNAPSHOT
The engagement in three parts.
01
Situation
Creator research and outreach took place inside mobile-first platforms with no suitable API, no machine-readable interface and variable latency through a remote-device window.
02
What we delivered
A packaged Windows application that captures profiles, reconstructs long screens, extracts structured context, evaluates campaign fit, drafts messages and supports controlled outreach and follow-up.
03
Our responsibility
We owned perception models, annotation tooling, the desktop application, orchestration and Windows packaging.
What shipped
One packaged Windows workspace covering research, evaluation, drafting, approval, execution and follow-up across three closed mobile platforms.
- Custom interface models and a full annotation, training and evaluation toolchain were delivered alongside the application.
- Most of the system can support another closed mobile workflow by adding new visual classes, state observations and a platform adapter.
They take ownership of the outcome, not just the task list. You feel they are accountable for getting the result across the line.
Stefan
Creator partnerships
Eyezilla operator feedback
CLOSED-INTERFACE ARCHITECTURE
Structured operation without an API or DOM.
Deterministic state and human approval surround probabilistic perception before any consequential external action.
01 · System layer
Rendered mobile interface
02 · System layer
Perception, OCR and state reconstruction
03 · System layer
Structured creator record
04 · System layer
Fit evaluation and draft
05 · System layer
Human approval
06 · System layer
Visually verified action
SELECTED SYSTEM EVIDENCE
Operating a workflow that only existed as pixels.
01
Visual access to interfaces that could not be integrated normally
Platform-specific object-detection models, OCR, template matching, geometry and screen-state logic identify the visible controls and content. Every observation is mapped back into current desktop coordinates rather than relying on one set of recorded click positions.
Why it mattered
The workflow could operate through the interface actually available to the client.
02
Complete profile reconstruction and evidence-backed evaluation
The application joins several scrolled captures while accounting for fixed interface chrome, floating elements, animated media, inconsistent movement and missing overlap. Structured language-model tasks then extract campaign-relevant facts and evaluate the profile against configurable criteria.
Why it mattered
Recommendations were based on the creator’s complete context and retained supporting evidence.
ENGINEERING DECISIONS
Preventing uncertain perception from becoming unsafe action.
Make perception hybrid rather than model-only
Small visual controls, text, landmarks and geometry do not all require the same tool. The application combines object detection, OCR, templates, image comparison and deterministic platform rules behind one observation layer. This reduces dependence on any single imperfect signal.
Visual automation
Measure latency by action
Remote sessions respond differently to taps, scrolls, navigation and typing. The application calibrates action-specific timing and waits for the screen to change and settle instead of applying one large fixed delay. Long waits remain cancellable.
Reliability
Resolve identity in stages
Names, ages, preview content, transcript overlap, perceptual signatures and optional visual identity evidence are evaluated in sequence. Weak OCR is not enough to attach private history to the first plausible record. Ambiguous cases enter a recovery flow.
Data integrity
RESPONSIBILITY
Responsibility stayed explicit
Client / domain team
- Campaign criteria
- Operating access
- Acceptance examples
Tested Works
- Perception and annotation
- Desktop application
- Orchestration and packaging
Shared decisions
- Approval boundary
- Exception policy
- Action acceptance
Packaged Windows workspace · acceptance against real operating constraints
SERVICES USED
SELECTED TECHNOLOGY
PythonPySide6RF-DETRYOLOPaddleOCROpenCVInsightFaceONNX RuntimeLLM APIsPyInstaller
RELEVANT EXPERIENCE
Relevant if your situation includes…
You need to automate a workflow inside software that exposes no suitable API.
You need AI to interpret complex records while deterministic code controls policy and action.
You need human review, identity safeguards and verifiable execution around consequential automation.
RELATED WORK
Two adjacent examples.
TESTED WORKS INTERNAL SYSTEM
Building an evidence-led account intelligence workflow.
Sixteen coordinated workflows with a shared database and operator-controlled LinkedIn actions.
View case study
ZAPFOLDER
Building an AI-assisted review and approval platform.
Live B2B SaaS with sharing, versioning, approvals, billing and production AI operations.
View case studySTART A CONVERSATION
Is an important workflow trapped inside a closed or awkward interface?
We can assess whether visual automation, integration or a hybrid approach can make the process reliable enough to operate.