In the context of surgical procedures performed in operating rooms, the anesthesiologist is responsible for manually documenting the administration of each medication administered to the patient. This task becomes particularly critical during urgent operations, where time is of the essence. The dynamic and often chaotic nature of the operating room environment, coupled with patient-specific constraints such as sensitivities and allergies, underscores the importance of accurate and timely medication documentation.
To address this challenge, we have developed an automated documentation system powered by an object detection unit. Our system simultaneously captures input from two cameras, extracts frame-by-frame the objects present in each frame, tracks these objects, and, based on predefined assumptions, maintains a record of each medication administered to the patient, including its type and administration time.
The project itself is composed of several modules, each utilizing established models. The Object Detection Unit employs a fine-tuned MMDetection model trained on a custom-built dataset. The Tracking Object Unit utilizes ByteTrack, a robust multi-object tracker that effectively handles objects with low detection scores, such as occluded objects. The Crossing Objects Unit and Documentation Unit are currently under development.