dionlahh/drone-detection-android-application

Drone Detection Map

Drone Detection Map is the Hunter Concept's Ground Sensor application.

It was built using Android Studio SDK, Flutter framework and supporting libraries.

Developed by myself, as well as vintageyj.

Drone Detection Map is designed to enable aerial surveillance of a deployment site through object detections on unauthorised (hostile) drones.

Features that Drone Detection Map offers:

Map view of current position
Area of coverage
Feature plotting
Object Detection with Hotswappable Inference Models
Range & Geopositioning Estimation on detected drones
Screen Mirror or Camera Feed via RTSP Stream

Prerequisites

Android Studio SDK
Flutter SDK
Android 9.0 and above

Integration
Features & Functions
a. Map
b. App Bar Buttons
c. Floating Action Buttons
d. Object Detection
e. Object Detection Setup
f. Red Line
g. Range & Geopositioning
Design & Architecture
a. BLoC
b. MVVM
Libraries

Integration

Due to outdated TFLite plugins for Flutter, the computer vision aspect of Drone Detection Map had to be built on native Android. As such, the Map module of had to be attached as a Flutter module.

When developing on different workstations, the Flutter packages and dependencies needs to be updated. This can be done so by running the following command lines within the Flutter module's working directory:

flutter clean

followed by

flutter pub get

Features & Functions

Drone Detection Map has mainly two feature-filled modules, the Map and Object Detector.

Launch

On application launch, it will first launch the map module, with orienation lock on. The screen will show the phone's current location and heading.

Map

https://raw.githubusercontent.com/dionlahh/drone-detection-android-application/HEAD/android/readme%20album/map/Updated/appbar_buttons.jpg

App Bar Buttons:

Orientation Lock Toggle
- True (default): Screen is locked to the phone's current position on the map, and and will automatically rotate towards the phones's heading.
- False: Screen is no longer locked, and the user can move the map around to view other parts of the map. Map no longer automatically rotate to the phone's heading.
Drone View Mode Toggle
- Last Detected Location (default): Draws a marker on the map, showing the last detection by the phone.
- Drone's Flight Path: Shows a polyline, drawn using the all the detected lattitudes and longitudes.
- Timelapse: Draws markers on the map, showing one detection at a time.
Grid Toggle
- Toggles on and off a grid on the map.
Map Layer Toggle
- Toggles between different map layers.
Launch Camera Button
- Switches the view to the camera for object detection.
Settings Button
- Allows the user to adjust the position marker's arc's radius and span using 2 different sliders. (default: 100m and phone's back camera's horizontal FOV)

Floating Action Buttons:

These buttons allows for the user to draw its corresponding shapes/objects on the map:

Clear All:

Button is only available when there are objects plotted on the map.
Deletes all plotted objects.

Polyline:

https://raw.githubusercontent.com/dionlahh/drone-detection-android-application/HEAD/android/readme%20album/map/Polyline_Trim_AdobeExpress.gif

Tap on the screen to plot the points of the polyline.
Save the plotted polyline by pressing the save button. This will change the polyline's colour to red, indicating that it has been saved.
Toggle button switches the opacity of all saved polylines between 1 (default) and 0.

Polygon:

https://raw.githubusercontent.com/dionlahh/drone-detection-android-application/HEAD/android/readme%20album/map/Polygon_Trim_AdobeExpress.gif

Tap on the screen to plot the points of the polygon.
Save the plotted polygon by pressing the save button. This will change the polygon's colour to red, indicating that it has been saved.
Toggle button switches the opacity of all saved polygon between 1 (default) and 0.
Saved polygons can be edited by tapping on the polygons
- Square markers on the polygon can be dragged to change the vertice's location.
- Circle markers on the polygon can be dragged to create a new vertice.
- Saved polygons can be deleted by tapping on the circle, followed by the delete button.

Markers:

https://raw.githubusercontent.com/dionlahh/drone-detection-android-application/HEAD/android/readme%20album/map/Marker_Trim_AdobeExpress.gif

Tap on the screen to plot different markers.
To add text beneath the markers, type the text in the textfield at the botttom of the screen. All the markers currently being plotted will have the same text.
Save the plotted markers by pressing the save button.
Toggle button switches the opacity of all saved markers between 1 (default) and 0.
Saved markers can be tapped on for more information regarding the marker's position

Circles:

https://raw.githubusercontent.com/dionlahh/drone-detection-android-application/HEAD/android/readme%20album/map/Circle_Trim_AdobeExpress.gif

Tap on the screen to plot a circle. (Only one circle can be plotted at a time)
To adjust the radius of the circle, use the slider at the bottom of the screen. The radius in meters is shown beside the slider. (Max: 500m)
Save the plotted circle by pressing the save button.
Toggle button switches the opacity of all saved circles between 1 (default) and 0.
Saved circles' radius can be adjusted by tapping on the circle on the map
Saved circles can be deleted by tapping on the circle, followed by the delete button.

Other Map Features:

Scale Bar
- Changes according to the current zoom of the map
Compass
- Rotates according to the phone's heading

Object Detection

The objection detection module is accessible by clicking on the Launch Camera Button.

https://raw.githubusercontent.com/dionlahh/drone-detection-android-application/HEAD/android/readme%20album/detector/detector_setup.gif https://raw.githubusercontent.com/dionlahh/drone-detection-android-application/HEAD/android/readme%20album/detector/od_setup.jpg

Object Detector Setup

Detection Switch
- Toggle for the Object Detection to be on or off (default)
Red Line Switch
- Toggle for the Red Line function to be on (default) or off
Calibration Switch
- Toggle for Calibration Mode to be on or off (default)
- Calibration Mode is used to calibrate the range estimator by setting the bounding box of a drone at a distance interval
Confidence Threshold Slider
- Tune the confidence threshold for detected results to be appear (default set to 30%)
Model List
- Select the custom trained model for the operation's requirement
Processor List
- Select which processor unit (CPU/GPU/NPU) to run the model on

Red Line

Drone Detection Map has a Red Line feature where it can establish a detection cutoff area on the screen. This prevents any unwanted detections below the Red Line, which can be useful in the future for the Hunter Concept's Interceptor drone. This feature can be toggled on or off in the settings page.

https://raw.githubusercontent.com/dionlahh/drone-detection-android-application/HEAD/android/readme%20album/detector/redline_demo.gif

Range & Geopositioning

The object detector module also comes equip with range and geopositioning capabilites.

https://raw.githubusercontent.com/dionlahh/drone-detection-android-application/HEAD/android/readme%20album/detector/drone_straightline_large.gif

The distance between the phone and the detected drone is estimated using the bounding box size of the drone relative to its actual size, as well as the Pinhole Camera Model, enabling the 2D screen coordinates to 3D world coordinates conversion.

The bounding box's screen coordinates returned by the inference machine is with respect to the input image size, so additional scaling steps has to be done to factor for the model's input image size and the screen's physical size.

Finally, using the derived distance, the phone's orientation, hearings, current position and the drone's relative heading, we can find the resulting latitude and longitude (LatLong) of the detected drone.

The resulting LatLong can then be saved during detections to be plotted on the map module.

https://raw.githubusercontent.com/dionlahh/drone-detection-android-application/HEAD/android/readme%20album/detector/drone_last_location.jpg https://raw.githubusercontent.com/dionlahh/drone-detection-android-application/HEAD/android/readme%20album/detector/drone_flight_path.jpg https://raw.githubusercontent.com/dionlahh/drone-detection-android-application/HEAD/android/readme%20album/detector/drone_flightpath_timelapse.gif

Clicking the Drone View Mode Toggle cycles through the different view for the detected drone's locations.

Design & Architecture

We used BLoC to implement the MVVM architecture for the map.

BLoC

Two different BLoCs were used. One for the application to decide which set of buttons and sliders to render on the screen, and when should it be rendered, and the other was used for handling on screen taps.

New functions and features to be easily implemented in the future, just create new events & states within the bloc, and specifiy the widgets to only render when it the BLoC is in the new state.
Testing and maintenance can also easily be done by tracking the module's state and events.

MVVM

We have four main models used, namely Circles, Polygons, Polylines, and Markers. Each of these models are then controlled by a view-model, ObjectMgr.

The models are not to be confused with the same objects within Flutter Map itself.
Flutter Map's objects are rendered based on the models that we have created.

If the user plots a new circle on the map, the model Circles is first created using the View-Model, then the map renders the CircleMarker based on the attributes on the Circles model.

Libraries

This section includes useful libraries and plugins used within the Ground Sensor project.

Flutter Dependencies:

Native Android Libraries: