We nicknamed our dual-camera Xperience prototype “Krabs” for its crab-like independent eyes and Mr. Krabs-like snarky personality (of SpongeBob SquarePants fame).
Why two servo-controlled cameras on a servo box rather than just one?
In DEMO-1, we tested our “one-eyed” rig, now named “Stuart” after a certain optically-focused and playful Minion. Further testing of Stuart after DEMO-1 showed that intermittent issues with wireless internet connectivity and the internet-of-things (IOT) platform controlling the servos limit the reliability of tracking full-court basketball action using one camera. The remote camera operator cannot always rotate the camera fast enough to keep the action in the middle of the video frame.
We investigated using a camera with a fixed wide-angle lens to cover the whole court, but the optical quality and resolution left much to be desired, e.g. player uniform numbers often were hard to read, impeding accurate color commentary. We also looked at how other video systems, such as BallerTV, combine a fixed wide-angle lens with AI-controlled digital zooming to track the ball, but found that tracking inaccuracies, zooming issues, and other deficiencies prevent achieving the entertainment experience we seek to deliver. We will continue to seek off-the-shelf solutions as alternatives to building our own rigs from scratch. We learn much by doing so.
For DEMO-2, we virtualized an approach we successfully used to capture game video on-site during pandemic attendance restrictions. One camera points at the left court and a second at the right court. Using video switching and streaming software on a mobile device wirelessly connected to both cameras, the on-site camera operator toggles the stream between the two fixed cameras as the game action moves between both halves of the court.
DEMO-2 Log
We sought to test how reliably Krabs could track full-court action connected through a 5G WiFi hotspot to a remote camera operator switching between the cameras using virtualization software and internet cloud services. We also tested the streaming video quality and our ability to add game action color commentary provided by a remote commentator, e.g. “What a block!” or “Clutch…nothing but net!”. Special thanks to the team at LifeTime Fitness of Oak Brook, IL for hosting us!
The Xperience video system architecture is as it was for DEMO-1:
The offsite operator remotely orients and zooms each camera to provide the optimal view of each half of the court, regardless of how near or above the court Krabs is placed. For instance, the tripod for Krabs could be placed 5 feet from the court midline, or 20 feet to the side and 15 feet up in the stands. This flexibility is important because different gyms and tournaments vary in where a tall tripod can be reasonably or safely placed.
We combined Krabs with multiple off-the-shelf software products and internet cloud services to deliver the live stream with sample color commentary. The operator and commentator were on-site, but the laptop and mobile devices they used were NOT connected to the 5G WiFi hotspot used by Krabs, but rather to the gym’s public WiFi network. Therefore, the operator and commentator, as if they were at their homes, were connected to Krabs and the live video stream over the open internet.
We made a quick rough-cut video of the DEMO-2 setup:
“What doesn’t kill you, usually succeeds in the second attempt.” – Mr. Krabs (SpongeBob SquarePants)
Fortunately, Krabs the Prototype escaped Mr. Krabs’ prognostication!
Krabs successfully live streamed full-court basketball game action while being entirely remotely controlled and monitored over the open internet, connected only through a 5G WiFi hotspot:
What's Next
- Capture a view of the scoreboard (score, time left, quarter) and integrate it into the live stream.
- Diagnose why sometimes the electronic chips driving the servos appear to be disconnected from the internet cloud service which enable the remote controls.
- Design and build a new printed circuit board and upgrade software to remotely control up to 8 servos (up to 4 cameras) from a single electronic chip. This would allow Krabs and 2 Stuarts to run off a single chip and deliver additional camera angles.
Company Update
We are up to 5 teens and about to add a 6th! We are delayed in engaging teams, clubs and tournaments for pilot opportunities, but look to catch up before the Christmas holidays. Broadcasting 100+ league and tournament games is not feasible in Q4, as we had hoped. However, the silver lining is we now have more time for product development and testing to act upon the learnings from DEMO-1 and DEMO-2 and deliver a better viewing experience with a simpler on-site setup process.