Sandvik Coromant · data engineering
One parser for two languages
A C# gRPC service with a Python client wheel that opens production's binary recording format to notebooks and Azure ML, so analysis and product read data the same way.
The problem
Production reads its sensor recordings through a .NET data model. Data scientists work in Python, and had to re-implement the parsing of the proprietary binary format themselves: two parsers for one format, maintained by different people.
The bridge
- A C# gRPC server wraps the production data model and maps it to protobuf messages.
- A Python client wheel gives notebooks and scripts the same objects, read by the same code the product uses.
- The wheel also runs inside the Azure ML environment, so the training pipeline parses data exactly as production does.
Before the bridge, I wrote a command-line parser, a first Python package and an Excel exporter. Each one taught me where the format's edge cases were, and convinced me that the answer was one source of truth rather than a better copy.