As drones continue to find use in a wide range of commercial applications, the focus is shifting towards higher levels of automation and tighter integration of drones with business processes, for significantly improved efficiencies. Recent advances in AI have enabled computers to make sense of the visual data around them, almost reaching human-level performance in some cases. Some of the tasks enabled by these algorithms include:
● Object detection – identify and locate objects of interest in an image
● Object counting – identify and count objects of interest in an image
● Image segmentation – classify pixels in an image into multiple finite segments to simplify the representation
● Change detection – detect changes between two temporally spaced images
● Image classification – classify an image into one of the known categories of images
The technological potential of drones is being further enhanced by combining autonomous drone tech with AI. Computer vision systems, mounted on drones, enable them to gather rich visual data either in the form of photos or videos. Processing this data using AI unfolds unique perspectives and information, which otherwise would be either impossible or very expensive to derive using traditional techniques involving human effort.
With the vision to leverage AI for drone applications, FlytBase platform is being further extended to incorporate AI capabilities to process aerial image data.
Workflow
FlytBase AI platform is based in the cloud, wherein the entire workflow of preparing datasets, training models and deploying trained-models for inferencing has been automated. This enables quicker turnaround time and faster iterations when a use case is being worked upon. Being in the cloud also helps in scaling the system up at runtime when demand (either for training, or for real-time inferencing) increases.
Examples of use cases that can be automated with FlytBase AI platform, are:
a. Object counting – e.g. counting the number of Arabian Oryx from an orthomap image. These are an endangered specie and keeping a tab on their count goes towards their conservation.
b. Object detection – e.g. locating cracks and rust areas from an image of industrial structures.
c. Change detection – e.g. detecting changes between two photos of a parking lot taking from almost the same vantage point at different times.
To harness FlytBase AI platform capabilities, customers bring in their use-case to FlytBase, along with sufficient training images dataset. The customer provided data is carefully cropped, labelled and packaged for training purposes, and added to an Image Dataset Library.
The FlytBase AI model-training workflow consists of:
● Model Library: Hosts object detection models to choose from during training
● Pre-trained weights library: Hosts weights from previously trained models to borrow representation from
● Image dataset library: Hosts packaged datasets provided by customers. The raw data is pre-processed for image augmentation and labelling before putting into this library
Via the above workflow, user can select various pieces of the training pipeline and initiate training on one of our GPU enabled cloud compute nodes. This results in a trained model ready for inferencing.
Once our model is trained, it is deployed on the platform for direct use by our users. Users can do live inferencing either via our web console or by using REST API’s exposed by the platform. REST APIs have the added advantage of integrating this platform with customer’s
system for further automation.
FlytBase AI platform is designed to support multi-tenancy, which enables utilisation usage of resources, and hence cost savings for our customers.
Deep Learning Algorithm
At the heart of the image-processing pipeline are state-of-the-art CNN models employing recent advancements in computer-vision and deep-learning.
Over the last few years, several object detection models have been published, which have significantly improved upon the previous generation, in terms of accuracy and speed of inferencing. Notable are, SSD, DetectNet, Fast R-CNN, Faster R-CNN, Yolo and Yolo V2.
Similarly, for image classification, ResNet50, VGG16/19 and Inception models are some of the most preferred models. Some models have better accuracy, while others might be faster at inferencing than others. Selection of a model takes into account these criteria, tailored to customer’s use case.
The pipeline allows several model implementations (same model with different hyperparameters or different models altogether) to be trained on the same dataset, simultaneously, so that the best can be chosen. Since different model implementations might need datasets to be arranged in different formats (e.g. from PASCAL VOC to TFRecord format), we have built adapters to transform the data on the fly to suit the model.
We have used transfer learning to tune the off-the-shelf pre-trained models for getting higher accuracy for detecting our object(s) of interest. This involves removing layers of the off-the-shelf pre-trained models to keep the correct level of representation from the previous dataset, before training them on new dataset.
The FlytBase AI platform is agnostic to the particular framework in which the models are implemented (Tensorflow, Caffe, Theano etc. ), by virtue of an abstraction layer. This allows the platform to assimilate the best implementation of cutting-edge models coming out of research labs, with ease.
Challenges and Solutions
Using high resolution aerial images to train computer vision models poses unique challenges:
a. Lack of sufficient training data: There are plenty of open training datasets out there, but almost all of them have images taken from human eye level. What makes aerial images unique is their top-down view of the objects. Moreover, for custom object detection, customers don’t often have enough images to train the model on, wherein we have to make do with limited set of images.
b. Very high resolution of the images: Computer vision models can process images of limited resolution at a time. For high-resolution images, we need to crop the images into sizable chunks and run inference on them one at a time. This can lead to double counting or misses.
c. Shallow features of objects: When looked down from the top, objects can have very generic shapes which a) can be hard to detect and b) can appear to be similar to other objects.
FlytBase AI platform uses various approaches to address these challenges, including data augmentation, cropping with different offsets for hi-res images, and training models on similar looking objects for better differentiation. Improving algorithms to address these challenges is a continuous process, further enriching the platform.
A Case Study: Arabian Oryx Detection and Counting
The Arabian oryx or white oryx is a medium-sized antelope native to desert and steppe areas of the Arabian Peninsula. It was extinct in the wild by the early 1970s, but was saved in zoos and private preserves, and was reintroduced into the wild starting in 1980. Keeping an upto date record of their population, with geotags, is an essential part of their conservation. The practice has been to count them manually by looking at a high resolution
orthoimage of the sanctuary. This is error prone and time consuming. Moreover, our customer had a backlog of such images taken over time, where the counting had to be done on all of them.
At FlytBase, we developed an image processing pipeline to automate the task of detecting oryxes in an ingested image and count them.
We started with 400 images which constituted an orthoimage, and extracted 1000×600 sized images, which had either oryx in them, or oryx like objects. These were then labeled and packaged into training and testing datasets in the Pascal VOC format.
A Faster R-CNN based object detection pipeline was set up in the cloud using the tensorflow object detection library. In the pipeline, the images were augmented by horizontally flipping and random resizing. Once everything was in place, the model was trained for 10k iterations.
The model so prepared could scan a 1000×600 sized image for Oryx. But it had to run on a high resolution orthoimage (29200×24160 pixels). To meet that requirement, the orthoimage was split into sections sized 1000×600. Detection was executed on them, individually, and the results were stitched back. To avoid double counting or misses, the split-detect-stitch procedure was repeated with different offsets for the splits, and the
median count of all the runs was obtained.
The results from this model were quite accurate and impressive. The FlytBase AI platform is able to process aerial image-data, gathered over several months, in order of minutes. This was the world’s first application of machine learning on drone image-data for Oryx
detection in the desert.
The Road Ahead
There is a vast potential to be unlocked for our customers, from the images they collect via drones. With its scalable architecture, automated pipeline, and with our vast experience in dealing with drones, their data and automation, FlytBase AI platform will result in significant improvement in efficiencies for our customers.
FlytBase AI platform is optimised for interpretation of drone data, and it seamlessly integrates with the rest of FlytBase platform to offer connectivity with your business applications. If you are looking to leverage machine-learning technology for automation of
If you are looking to leverage machine-learning technology for automation of your drone data-processing, please reach out to our experts at flytbase.com/ai
]]>
The drone racing world just leapt over a huge hurdle when it comes to popularizing the sport of drone racing. The Drone Racing League on Tuesday announced the launch of its first-ever 5G DRL racing drone, which makes it possible to live stream high-definition racing footage. Now, fans can see the pilot’s first person view racing footage in real-time over a high-definition feed with minimal latency or delay.
Dubbed the Magenta 5G drone, it marks one of the first racing drones in the world to have an embedded 5G module capable of live streaming video directly to the Internet. 5G is largely seen as crucial to popularizing drone racing and making it more accessible by allowing all spectators views of the dizzying, FPV views that racing drones can show.
With a dual-FPV and HD-streaming camera system, the drone will be able to film mile-long drone racing courses, made possible through the 5G connection as well as its 5s lipo battery setup for extended flight time. And while the drone won’t compete with the zippiest racing drones, it can still fly more than 60 miles per hour.
The 5G component of Magenta is done through a partnership with wireless network provider T-Mobile. DRL Pilots currently fly via analog radio transmissions. While that allows for lower latency, there’s a trade-off in the technology; they sacrifice crisp quality footage in their goggles. But as 5G technology improves, pilots will be able to see high-quality, crisp FPV footage in their goggles with low latency. For fans, it means the ability to experience FPV clips on their own mobile devices to gain that same sensation that they are flying inside the drone in real-time.
The deal with T-Mobile to bring higher-quality drone racing footage to fans could potentially be a huge win for DRL, which is already seen as a leader in drone racing. DRL is a global, professional drone racing property, consisting of all aspects of the FPV and drone racing lifestyle, including putting on worldwide events, selling simulators for folks to practice on gaming devices at home, creating its own custom-built racing drones and broadcasting races on TV. Since DRL was created in 2015, it has raised millions of dollars in funding from investors including Hearst Ventures, Muse lead singer Matthew Bellamy, and Miami Dolphin’s owner Stephen Ross’s venture-capital firm RSE Ventures.
Why telecom giants are jumping on 4G and 5G
Many telecom giants are rapidly jumping into not just being leaders in 4G and 5G, but trying to lead when it comes to 4G and 5G in drones.
Just last week, French drone maker Parrot announced an exclusive partnership with T-Mobile competitor Verizon to bring the first out-of-the-box, Verizon 4G LTE connected drone solution to the United States. That means that now you can fly with your Parrot ANAFI Ai drone from anywhere there’s a signal with near real time data transfer.
And generally speaking, 5G is considered important for complex drone operations in ensuring a better drone connection, which is important for both safety in ensuring no flyaways or crashes, and also usefulness in allowing faster transfers of larger datasets. Any drone flight beyond the operator’s line of sight — including multi-mile energy infrastructure inspection, first response to find lost hikers in the woods or police activity — requires a reliably strong signal. A lost connection could be detrimental, typically resulting in flyaways or crashes.
But T-Mobile is one of the first and few to hone in on drone racing.
“Drones are one of the most compelling use cases for 5G and we’re working towards a future where all drones will eventually be 5G-connected – that’s why we’ve teamed up with DRL, to fuel this innovation,” said Neville Ray, President of Technology at T-Mobile.
And sure, it’s unlikely 5G racing drones are saving lives by finding lost hikers in the woods. But 5G and drone racing go hand-in-hand as the new 5G-enabled drone could redefine sports entertainment, capturing exhilarating, crisp video footage.
“The Drone Racing League is a perfect case study for showcasing the benefits of T-Mobile 5G wireless technology with our high-speed racing drones,” said DRL President Rachel Jacobson. “Our fans love innovation and discovering how new technology is developed, and we know our 5G-enabled drone will get them excited about new ways they will be able to experience the immersive thrill of professional drone racing.”
See a little bit more about how the 5G DRL racing drone was made in the video (created by T-Mobile and DRL) below. The short documentary covers DRL engineers hand-building an initial prototype of the drone in the DRL lab in New York City, testing it for the first time and more:
What to expect from the 5G DRL racing drone
You can see the 5G DRL racing drone in action this week at the MLB At Field of Dreams in Dyersville, Iowa. No surprise, the event is sponsored by T-Mobile. As part of the event, the Magenta 5G drone will zip through nooks and crannies of the stadium, giving fans a largely never-before-seen, behind-the-scenes view of the iconic ballpark and movie site. The drone will fly from the cornfields to the set of the Field of Dreams house to the original ball field, and then on to the MLB field where the long-awaited Field of Dreams Game will be played.
After that, DRL said it intends to feature the 5G Magenta drone through its 2021-22 season at various DRL and T-Mobile events. While it won’t necessarily race, the drone will fly around the course and film previews of the three-dimensional race tracks during and ahead of the actual races.
DRL also said it intends to improve the Magenta 5G drone in future iterations, including allowing the 5G model to connect to the drone’s command and control functions to enable flight over T-Mobile 5G.
The post 5G DRL racing drone dubbed Magenta is huge leap for FPV racing and beyond appeared first on The Drone Girl.
]]>