Hands-on lab to demo how to store, process, read, analyze video on Streaming Data Platform
Components
- Streaming Data Platform: Streaming Data Platform allows for harnessing their real-time and historical data in a single, auto-scaling infrastructure and programming model.
- Pravega: Pravega provides a new storage abstraction - a stream - for continuous and unbounded data. A Pravega stream is a durable, elastic, append-only, unbounded sequence of bytes that has good performance and strong consistency. Pravega provides dynamic scaling that can increase and decrease parallelism to automatically respond to changes in the event rate.
- Flink: Apache Flink® is an open-source stream processing framework for distributed, high-performing, always-available, and accurate data streaming applications.
- Docker: This demo uses Docker and Docker Compose to greatly simplify the deployment of the various components on Linux and/or Windows servers, desktops, or even laptops.
- GPU: GPUs are essential for increased performance of processing data.
- CUDA: To utilize the GPUs, NVIDIA CUDA libraries are required. CUDA 10.0 is used in the project.
- Tensorflow: The core open source library to help develop and train ML models.
Demo environment
Workload Flow
- Camera recorder collects the video feed into a stream
- Object Detector job collects the input video stream, and returns output from running on the pre-trained model to detect and identify objects detected into a stream
- Video Player displays the output video stream with bounding boxes and labels on detected objects
