Back to projects

Semantic Similarity of Textual Data

Architecture

Semantic Similarity of Textual Data architecture diagram
The C# worker runs in Azure Container Instances. Storage Queues trigger experiments, Blob Storage holds inputs and outputs, Table Storage records metadata, and the OpenAI API supplies embeddings.View full-size diagram ↗

Research focus

Evaluated whether embedding-based similarity remains reliable under controlled spelling errors and changes in passage length, using unrelated text as a baseline.

Implementation

Implemented a C# worker on Azure Container Instances. Queue messages trigger experiments; the worker reads inputs from Blob Storage, calls the OpenAI API for embeddings, computes cosine similarity, and stores results and metadata in Azure Storage.

Findings

Related passages consistently scored above unrelated controls across six tested text lengths. Similarity declined as typos increased, but remained above unrelated-text scores even at the most severe tested level.

Technology

C# · Azure Container Instances · Azure Storage · OpenAI API · Docker

Selected results

Line graph showing similarity decreasing as typos increase, with unrelated text much lower
Typo robustness. Similarity declined as spelling errors increased, yet remained well above the unrelated-text controls.
Line graph comparing same-topic and unrelated text across six text lengths
Text-length sensitivity. Across six text lengths, same-topic passages stayed substantially more similar than length-matched unrelated text.
Let’s talk about your project