Operation Last-Mile: Critical Drone Delivery Testing Report

Table of Contents 1. Overview

2. Executive Summary 3. Location 4. Aircraft and Camera 5. Payload and Drop Mechanism 6. Aircraft Payload Capacity

7. Team Participants 8. Communications 9. Test Plan 10. Schedule of Operations

11. Testing Results 12. Lessons Learned 13. Flight Data Analysis 14. Recommendations for Staging a Real Life Delivery Operation 15. Appendix

About DroneUp DroneUp provides complete drone solutions that include FAA compliant consulting services, Part 107 pilot flight services, aerial data collection & processing, data delivery & analysis, training, program integration, and equipment sales to commercial industries and public sector organizations. ​ DroneUp operates globally with more than 10,000 certified drone pilots. ​ DroneUp is headquartered in Virginia Beach, Virginia, and a SWaM or Small, Women-owned, and Minority-owned Business certified as a small business by the Commonwealth of Virginia. For more information: ​ droneup.com ​ .

160 Newtown Road, Suite 302 | Virginia Beach, VA 23462 | 877-601-1860 | ​ droneup.com

2

1 Overview

In response to the COVID-19 Pandemic of 2020, various government agencies began inquiring about the capabilities of drones to deliver critical medical and other supplies to quarantined areas in the event the pandemic forced the physical quarantining of entire towns and cities. In particular, members of the commercial drone community to document their capabilities through surveys of drone manufacturers and drone service providers. Based on previous experience delivering payloads with the DJI Inspire 2 drone, ​ DroneUp LLC ​ , a leading drone service provider in the United States, responded that it could assemble a team of 50 drones and pilots who could provide delivery capability in the Eastern half of the United States. At the same time, ​ DroneUp ​ proposed to the Commonwealth of Virginia conducting a test of these capabilities through an exercise to be held at an outdoor test environment. The purpose of the test would be to determine the limits of drone delivery for small packages in a quarantine environment. Representatives of the Commonwealth of Virginia’s Center for Innovative Technology (CIT) were intrigued by ​ DroneUp ​ ’s proposed exercise and conducted a search for an appropriate location to perform this exercise. Of three viable locations, the former campus of Saint Paul’s College in Lawrenceville, Virginia was selected. The 55-acre campus was under the stewardship of Brunswick County, Virginia. Various local, regional, and state entities agreed it was an appropriate location to perform such testing because of its rural location, variety of terrain, buildings, mature trees, and the ability to be secured for the protection of the general public during the testing. Once funding from the Commonwealth was approved and it was concluded that the operation could be held safely by complying with recognized virus protocol, the exercise was scheduled. It was called, ​ Operation Last-Mile: Critical Drone Delivery Testing. DroneUp ​ scripted a 2 ½ day test plan to utilize the ​ DroneUp pilot network ​ and readily available, industry-standard drones to simulate the delivery of 1.275-pound payloads of medical and other critical supplies in city-like conditions. This exercise sought to test the capabilities, capacity, and scalability of small package drone delivery while establishing a baseline of performance for multiple drones operated under FAA Part 107 rules. Testing variables included delivery over buildings, trees, and power lines over varying terrain. Tests were designed for both single aircraft and simultaneous multiple aircraft operations. Also tested were semi-autonomous flights versus manual flights, aircraft deconfliction using command center drone visibility software versus visual command only, and night operations.

160 Newtown Road, Suite 302 | Virginia Beach, VA 23462 | 877-601-1860 | ​ droneup.com

3

2 Executive Summary

This exercise proved that Part 107 pilots using commercially available drones could successfully and repeatedly deliver 1.275-pound payload packages to 10-foot diameter targets on 1500-foot round trips under FAA Part 107 flight rules. Over 200 total flights were flown during the day and at night. Test plans called for 90 progressively more difficult delivery test flights. Those 90 delivery flights were recorded in rich detail. Of the 90 delivery flights, 90 payloads were dropped inside a 10-foot diameter, predetermined target. One delivery attempt was safely aborted when the wind caused instability, that flight was then reflown successfully. Four delivery teams operated during the operation. Three ​ DroneUp ​ teams operated the DJI Inspire 2 drone. Each ​ DroneUp ​ team consisted of a Remote Pilot in Command (RPIC), a LoadMaster, and a Visual Observer (VO). The RPIC manned the sticks on the controller. The LoadMaster was adjacent to the RPIC and operated the radio and managed the payload. The VO was positioned in the target area to ensure a visual line of sight was maintained on the aircraft at all times. A fourth pilot team from UPS/Workhorse also participated in Operation Last-Mile. They flew tests independent of ​ DroneUp ​ and also flew alongside DroneUp ​ in Test 5, to test airspace deconfliction. The flight statistics reported include only the 90 DroneUp flights flown with the Inspire 2 drone. The average round-trip flight distance to the 19 different targets ranged between 452 feet and 1501 feet, averaging 924 feet. The average delivery elapsed time was 4.24 minutes, which consumed 20.2% of the aircraft’s battery power. Most of the targets for the payload deliveries were purposely placed close to obstacles such as trees, buildings, and power lines because the purpose of the testing was to find the limits of safe and efficient drone delivery. In the real world, wide-open targets would be selected, which would take an estimated 42 seconds off the average round trip delivery times. Taking into account delivering to wide-open targets, we estimate that 2000-foot round trip flights would, on average, take 5.4 minutes (including payload handling and battery swap-outs) and consume 30% battery. DroneUp ​ estimates that 3 flight crews of 3 members each could deliver between 150 and 300 packages in an 8-hour shift depending on the distance of the take-off spot to the target. Based on actual recorded performance, here are the projections we make for round trip deliveries at target distances between 500 feet and 2500 feet.

160 Newtown Road, Suite 302 | Virginia Beach, VA 23462 | 877-601-1860 | ​ droneup.com

4

Summary of Delivery Capacity 1.275-pound Payload on Inspire 2 Drone 3-Person Delivery Team (RPIC, LoadMaster, VO)

Target

Round

Take-Off,

Time

Time From Total

Del per

Del per

Distance

Trip

Payload / Landing, to Target

Target

Time

Team

3 Teams

Drop Time

One Way Distance Batt Time

@ 8mph @ 15mph Per Del Per 8 (Net 7) Per 8 (Net 7)

(feet)

(miles)

(min)

(min)

(min)

(min)

(min)

Hour Shift

Hour Shift

500

0.2

1.0

2.2

0.7

0.4

4.3

98

294

1000

0.4

1.0

2.2

1.4

0.8

5.4

78

234

1500

0.6

1.0

2.2

2.1

1.1

6.5

65

195

2000

0.8

1.0

2.2

2.8

1.5

7.6

56

167

2500

0.9

1.0

2.2

3.6

1.9

8.6

49

146

In addition to 90 recorded deliveries, successful demonstration test-flights included using drone thermal imagery to see humans and drones at night, using a drone as a public address system, showing that drone noise is not harmful to hearing and delivering a package to an automated drone package mailbox.

Important lessons learned include:

● The operating procedures for handing off the line-of-site responsibility between the RPIC and the VO can be better defined. ● The operating procedures for the use of radios can be streamlined. ● The terminology used by the VO to instruct the pilot on how to reposition the drone in the target area can be better defined. ● The use of checklists before and between flights can be refined and enforced. ● Providing lighting of some kind on drop targets at night might improve speed. ● Pre-programmed flights help ensure consistent flight parameters but take longer than manual flights. ● Providing the mission commander with a dashboard showing locations and video feeds from all active drones is helpful. DroneUp ​ is encouraged by the success of this operation. Not only did the pilots hit obscured and obstacle-ridden targets 100% of the time, but over 50 significant lessons

160 Newtown Road, Suite 302 | Virginia Beach, VA 23462 | 877-601-1860 | ​ droneup.com

5

learned were recorded which will help improve the safety and efficiency of drone deliveries. We look forward to leveraging what we’ve learned from Operation Last-Mile and testing delivery of potentially heavier payloads over potentially long distances.

3 Location

All tests were performed on the property of the former St. Paul’s College, located at 115 College Dr, Lawrenceville, VA 23868.

160 Newtown Road, Suite 302 | Virginia Beach, VA 23462 | 877-601-1860 | ​ droneup.com

6

4 Aircraft and Camera

The aircraft ​ DroneUp ​ chose for Operation Last-Mile was the DJI Inspire 2 outfitted with the Zenmuse X5S camera. This model was released in 2017 and is often used for cinematography. It features a retractable landing gear system that lends itself to reliable delivery when paired with the Skyzimir Stork 2 drop mechanism.

160 Newtown Road, Suite 302 | Virginia Beach, VA 23462 | 877-601-1860 | ​ droneup.com

7

The baseline Inspire 2 retails for $3299 and comes with a low-resolution first-person view camera. For this operation, we added the optional Zenmuse X5S (including gimbal) which retails for $2049. This high-quality camera ensured the pilots a good view of the drop areas to assess the area for safety before dropping the payload. This camera weighs 461g (1.01 lb). Another camera option for this aircraft is the Zenmuse X4S camera at 253g (.56 lb), but it had been discontinued by the manufacturer so it was not chosen for this test.

5 Payload and Drop Mechanism

The payload for each delivery totaled 20.4oz (578g or 1.275lb) consisting of a lightweight polypropylene bag, a ziplock bag of sand, a medicine bottle, a small cardboard box, an 8-foot-long paracord tether, and 2 S-carabiner clips.

160 Newtown Road, Suite 302 | Virginia Beach, VA 23462 | 877-601-1860 | ​ droneup.com

8

The assembled payload bag was connected to a DJI Inspire 2—a popular consumer drone—using an S-carabiner clip attached to a Skyzimir Stork 2 drop mechanism. The Stork 2 features a set of stainless steel bars that interlock when the Inspire 2 landing gear is in the down position. The drop mechanism on the drone allows the release of the payload when the pilot uses a manual switch on his controller to raise the landing gear. For each drop, the bag with a tether was left at the drop site.

160 Newtown Road, Suite 302 | Virginia Beach, VA 23462 | 877-601-1860 | ​ droneup.com

9

6 Aircraft Payload Capacity

The DJI Inspire 2’s (4) motors (model 3512) produce a combined thrust of 8500 grams. A general rule of thumb is that a drone should ideally hover at 50% throttle, therefore the manufacturer recommends an Inspire 2 maximum takeoff weight of 4250g. Each of the (4) Inspire 2’s in this exercise weighed 3957g, which included the aircraft, batteries, props, X5S camera, and drop mechanism. The aircraft weight at 3957g plus the payload at 578g (1.275 lb) created a total weight of 4535g. This weight represented 53.3% of the total thrust of the motor/propeller system. DroneUp ​ deemed that this 3.3% overage over the general rule of thumb was acceptable considering the nature of the flights (straight flights with little maneuvering), the supervision

160 Newtown Road, Suite 302 | Virginia Beach, VA 23462 | 877-601-1860 | ​ droneup.com

10

being provided (more than a dozen Part 107 pilots participating and observing) and the short-range of the flights (average 924 feet round trip). During the 2 ½ days of flying this drone with this payload, there was no indication of payload weight being a problem. There was one incident when the wind seemed to be an issue and that resulted in a safely aborted delivery shortly after takeoff. The Inspire 2 drone, having been in production for 3 years, is readily available across the United States. Within 300 miles of Washington, DC, ​ DroneUp ​ identified over (50) Part 107 commercial pilots within its network who indicated they owned this drone. DroneUp also received a commitment from its sister company in Norfolk, VA that committed to manufacturing 200 compatible drop mechanisms within 2 weeks of placing an order. For critical packages of up to 1.275 pounds, the Inspire 2 as outfitted for this operation would be ideal. If the X5S camera were replaced with the X4S at a savings of 208g, the resulting total weight as flown in these tests would represent 50.9% of the rated thrust of the motors/propellers. If the large camera were to be removed altogether and the pilot relied solely on the permanent low-resolution First Person View (FPV) camera, the 4250g of recommended payload capacity would allow an increase in the payload to 1.66 lb while staying at the “50% hover throttle” general rule of thumb.

7 Team Participants

DroneUp ​ queried its database of 10,000 Part 107 pilots to find the pilots within 100 miles of Lawrenceville, VA, and sent them each an email describing Operation Last-Mile and seeking their participation if they met the criteria below. All the candidates were located in either Virginia or North Carolina. They were paid a flat rate per day plus a meal and lodging per diem to participate. Interested pilots were instructed to respond to our email describing their relevant experience. These are the criteria we asked the candidates to meet. (1) ​ DroneUp ​ Call Sign (User Profile Name) (2) If not already submitted via the ​ DroneUp app ​ — your part 107 number (3) Number of hours as Pilot in Command (PIC) (looking for pilots with 75-100 hours as a PIC) (4) Number of hours piloting an Inspire 2 or Matrice 210 (looking for 25 hours in at least one of those drones) (5) Experience with night operations — yes or no. If yes, please verify your training as well as whether or not you possess a Part 107.29 waiver. (6) Any public safety background? (7) Any commercial UAV background? (8) Any scheduling conflicts being in Lawrenceville, VA the week of April 6 (9) Your current city of residence

160 Newtown Road, Suite 302 | Virginia Beach, VA 23462 | 877-601-1860 | ​ droneup.com

11

A total of 103 pilots were contacted via email and invited to respond. Thirteen respondents met the criteria and provided sufficient detail to be considered viable candidates. The top 10 candidates were invited to be interviewed via a video call with the ​ DroneUp ​ Flight Operations team. In the end, 8 applicants were chosen. Here is a summary of the qualifications of DroneUp ​ ’s Operation Last-Mile pilot team. ● All pilots had Part 107 certificates and were experienced, commercial drone pilots. ● Almost all had been pilots prior to the establishment of the Part 107 rules in late 2016. ● The average number of hours logged as a drone pilot was 1193. The mean was 165. ● Four of 8 had logged flight hours on the Inspire 2 or Matrice 200 drones. ● 2 pilots were current and 3 pilots were former public safety members, particularly with fire departments. At least one of the pilots not involved in public safety had a military background. So at least six of the eight pilots had either public safety or military background.

● Four of the pilots had Part 107.29 waivers to operate at night. ● The pilots ranged in age from 39 to 67 with the average being 50.

DroneUp ​ has a Part 107.29 waiver allowing it to operate outside the daylight operating period of between 30 minutes prior to sunrise until 30 minutes after sunset. In order for our pilot teams to operate under ​ DroneUp ​ ’s FAA waiver, each had to pass a training course in night operations. While four of our Operation Last-Mile pilots had their own Part 107.29 waivers, all 8 pilots attended a 2-hour night operations training course provided by the ​ DroneUp ​ Airboss on Monday, April 6. The course was held via webinar and all 8 pilots passed the 25 question test with an average score of 92% - allowing all participating pilots the opportunity to fly nighttime delivery flights on April 8. The pilots were divided into three teams (Yellow, Blue, and Green) with three members each. DroneUp ​ provided a Part 107 pilot as a VO for the Blue team. For each individual flight, the team roles were RPIC, LoadMaster, and VO. The RPIC piloted the aircraft and made the final decision on fly/no-fly, drop/no-drop, and abort/continue the flight. The LoadMaster was responsible for examining the payload, attaching the payload to the aircraft, ensuring the tether did not foul the propellers upon takeoff, and relaying radio communication to and from the RPIC. The VO was responsible for keeping the drone within line of sight once the RPIC lost line of sight and directing the RPIC during the descent and drop of the payload over the target. Because of the COVID-19 virus epidemic, ​ DroneUp ​ adhered to guidance from the Commonwealth of Virginia on social distancing and best practices in mitigating virus risk. A registered nurse was hired to screen all participants daily, (taking temperatures and interviewing) and the participants were given labels to wear indicating they had been screened. The Deputy Sheriffs assigned to provide security at the exercise entrance were instructed not to allow anyone entry who had not been screened.

160 Newtown Road, Suite 302 | Virginia Beach, VA 23462 | 877-601-1860 | ​ droneup.com

12

DroneUp ​ issued masks to every participant who didn’t bring one and made hand sanitizer readily available. The nurse monitored all activities and reminded participants and observers to observe social distancing. Every known precaution was implemented to ensure the safety of the exercise participants.

160 Newtown Road, Suite 302 | Virginia Beach, VA 23462 | 877-601-1860 | ​ droneup.com

13

8 Communications

Push to Talk (PTT) radios were provided to each member of the participating flight teams. This allowed communication with the Airboss, communication between RPIC, LoadMaster, and VO, and communication with other flight teams. Channel 1 was assigned to the Airboss, the ​ DroneUp ​ Operations Manager, and the ​ DroneUp CEO. Channel 3 was assigned to flight operations, particularly the communications between piloting teams and their VO’s in the field. Channel 5 was dedicated to facility logistics. Even channels were available to allow teams to have conversations without stepping on the communications of other teams. This convention was especially useful during Test 3 when multiple teams were flying and delivering at the same time.

The radio communication protocol was discussed during the project briefing and in each flight session briefing. The instructions included the following:

● Identify the party you want to talk to and then identify yourself prior to speaking your message. Example: “Green Team, Green VO. I’m in a position at target 2-1”. ● Keep radio messages concise. Think about what you’re going to say before you say it. Remember that while you are holding down the PTT button, only you can be heard across all the radios on that channel. ● Deliveries, take-offs, and landings take precedence on the radio. If you have a question or comment about something else, wait until the important communication has finished to key the PTT button. ● Under Part 107 rules, the RPIC cannot allow the drone to leave his visual line of sight until he receives confirmation that the VO has obtained a visual line of sight of the drone. So we established that the VO was to announce his visual acquisition of the drone as soon as he had acquired it and the RPIC was not to fly further until he had that confirmation. ● Methods used by the VO for directing the RPIC to the correct location above the target were varied. Some said, “Go forward 5 feet.” Others said “Slide right.” and then “Hold.” if the correction was more than a few feet and “Bump right.” if the correction was slight.

Role of the Airboss and Communications

DroneUp ​ provided the Airboss for Operation Last-Mile. The Airboss was in charge of directing all flight takeoffs and landings and overall direction regarding weather, wind, which tests were to be run, when to break for meals, etc. Delivery drops were managed by the flight teams and their VOs without Airboss intervention. The directions from the Airboss came generally from the radio so that all personnel involved in the operation knew what was happening.

160 Newtown Road, Suite 302 | Virginia Beach, VA 23462 | 877-601-1860 | ​ droneup.com

14

The flight teams were adjacent to one another, which made verbal communication possible between the teams. ​ DroneUp ​ ’s (3) teams were about 20 feet from one another and each had a landing/takeoff square of 20 by 20 feet. The Workhorse team was located approximately 50 feet to the Northwest and down a slight hill from the ​ DroneUp ​ teams. All teams requested permission from the Airboss to take off or land and permission had to be granted before that action was taken.

9 Test Plan

The following section describes the plan, objectives, and scoring criteria of each test the delivery teams performed. In all the tests, the following scoring criteria and demerit system, if applicable, were followed. Scoring Criteria: ​ The test began when the ​ ​ pre-test briefing was complete. ​ ​ DroneUp ​ Flight Crews were tasked with pre-flighting their aircraft, loading the packages, delivering the packages, and returning safely. ​ ​ Human ​ ​ Data Recorders recorded the following during the operation:

160 Newtown Road, Suite 302 | Virginia Beach, VA 23462 | 877-601-1860 | ​ droneup.com

15

● The number of successful deliveries ​ . A successful delivery was defined as a package that came to rest within a 10-foot diameter around an 18-inch cross marking the exact center of the delivery location. ● Time to complete each delivery. ​ Time recording started when the package was loaded and stopped when the aircraft landed after returning. ● Time to complete all deliveries. ​ Time recording started as the preflight inspection was performed, and ended after the final landing. ● Fuel consumption. Each delivery ​ battery level was recorded at each takeoff, each delivery, and each landing. Demerit Points: ​ Demerit points were recorded for incidents that would gravely impact safety or operational efficiency in live operations performed outside of the testing environment. The occurrence of a single demerit point indicates an expected real-world emergency or catastrophic failure. Any of the following conditions would be recorded for each test as demerits: ● Controlled Flight Into Terrain (CFIT) - ​ Any portion of the aircraft contacts terrain in controlled flight, other than landing ● Takeoff or Landing Damage to Airframe - ​ Any damage to the aircraft occurs during takeoff or landing ● Inadvertent Activation of Emergency Procedures ​ Emergency procedures are activated by another mechanism than the RPIC, such as a C2 link failure ● Failure of Flight Critical Avionics - ​ An avionics component critical to flight safety fails (IMU) ● Failure of Powertrain - ​ A powertrain unit fails ● Near-Miss - ​ A “Near Miss” is defined as 2 aircraft in en route passing within 25’ horizontally or 10’ vertically of one another. Additionally, a “near miss” is any situation where an sUAS pilot must take evasive action to prevent midair collision with another sUAS. ● Viral Contamination - ​ Drone Descends below 8’ AGL. The CDC had provided guidance that if an object remained 10 feet above the ground, the chance of COVID-19 aerosol particles infecting the object was very low. We determined that a drone with an 8-foot tether dropping the payload 2-3 feet above the ground would avoid contamination in a simulated quarantine environment. Our targets were unpopulated at the drop times so the risk of infection at any height was near zero. But we wanted to demonstrate that the drone drop height could be successfully controlled using VO’s. ● Drop Mechanism Failure - ​ The drop mechanism fails ● Payload Entanglement - ​ The payload becomes entangled ● Loss of Communications between RPIC and VO

160 Newtown Road, Suite 302 | Virginia Beach, VA 23462 | 877-601-1860 | ​ droneup.com

16

9.1 TEST 1: Control – ​ Establish Performance Baseline Test 1 Objective: ​ Test 1 ​ ​ established the baseline performance of each drone model in a standardized test of a fixed number of deliveries in a fixed set of locations. This test was designed to control, as much as operationally feasible, the variables of environment, human factors, and objectives. The baseline performance of each drone in an idealized environment was used as the control group to compare the impacts of each on delivery performance on each successive test. Test 1 Description: ​ Each ​ DroneUp ​ flight crew, consisting of RPIC, LoadMaster, and VO, was tasked with delivering 5 packages, each weighing 1.275lb, to the (5) drop locations identified below. For this test only, the takeoff location was moved to allow visual line of sight to be maintained by the RPIC throughout the flight. The five target locations were all in the area called “the quad”. This test was repeated through 3 flight crew rotations, and the scores of each rotation were averaged to eliminate flight crew variance in the result. Only one flight team operated at a time and all 3 teams completed all 5 deliveries. The test results were recorded by Data Recorders assigned to that task by the Airboss. These Recorders were either trained Part 107 pilots or temporary workers hired locally who were supervised in their recording tasks by ​ DroneUp ​ Part 107 pilots not participating in flight operations. A sample recording sheet is included in the appendix. Test 1 Location: ​ This test was performed at the 5 drop locations shown in blue near the center of the map. The takeoff and landing area was located at the blue marker on the southeast corner of the map.

160 Newtown Road, Suite 302 | Virginia Beach, VA 23462 | 877-601-1860 | ​ droneup.com

17

Test 1 Resources: ​ 3 Inspire 2, ​ ​ 3 Flight Crews (RPIC/LoadMaster/VO) ​ , ​ 15 packages, 5 drop locations, communication radios. Test 1 Limitations: ​ For the purposes of test integrity, flight crews performed the drop function using only the stock DJI Inspire 2 and supporting hardware plus the Skyzimir Stork 2 drop mechanism. All Inspire 2 batteries were fully charged at the start of each test. No other software than DJI Go 4 was used in flight to complete Test 1. The RPIC, LoadMaster, and VO were all co-located for the duration of the operation and used verbal communication for Crew Resource Management (CRM). Test 1 End State: ​ The test was complete when each flight crew attempted one delivery (successful or unsuccessful) at each of the 5 defined locations for a total of 15 dropped packages. 9.2 TEST 2 E-VLOS Operations - ​ Determines variables in performance between Visual Line of Sight (VLOS) operations, and flights where terrestrial obstructions prevented VLOS during the package drop/delivery. Test 2 Objective: ​ The objective of Test 2 was to determine variables in performance between test operations within visual line of sight of the RPIC, and operations where VLOS of the RPIC is occluded by terrestrial obstructions. Additionally, this test validated the performance of the VO coordination with cross-trained RPIC/VO’s, and VO’s with limited training and experience. Test 2 Description: ​ Two ​ DroneUp ​ flight crews ​ ​ performed five deliveries to five locations sequentially. Each of the five delivery locations was positioned behind structures and/or obstacles in a manner to intentionally obstruct the visual line of sight of the RPIC. Each flight crew utilized a downrange VO communicating with the RPIC directly by PTT radio to meet the requirements of FAA rule 107.31(b)(2). The first flight crew consisted of (2) cross-trained RPIC/VO’s. We intended the second flight crew to consist of one RPIC and one layperson VO with minimal prior training, but that plan did not get executed. Instead, VO’s with Part 107 certificates were used.

160 Newtown Road, Suite 302 | Virginia Beach, VA 23462 | 877-601-1860 | ​ droneup.com

18

Test 2 Location: ​ This test took place at the locations indicated in green in the northeast corner of the map. The takeoff and landing area is marked by the green marker on the southwest corner of the map.

Test 2 Resources: ​ 2 Inspire 2, 2 RPIC/LoadMaster/VOs, 1 Layperson VO, 5 drop locations, communications radios Test 2 Limitations: ​ For the purposes of test integrity, flight crews performed the drop function using only the stock DJI Inspire 2 and supporting hardware, excluding the drop mechanism. All Inspire 2 batteries were fully charged at the start of each test. No other software than DJI Go 4 was used in flight to complete the operation. The RPIC and VO maintained direct 2-way communication at all times during the test. Test 2 End 2 State: ​ The test was complete when each flight crew attempted one delivery (successful or unsuccessful) at each of the 5 defined locations.

160 Newtown Road, Suite 302 | Virginia Beach, VA 23462 | 877-601-1860 | ​ droneup.com

19

9.3 TEST 3: Deconfliction – ​ Validate the performance of command and control solutions and automated or manual deconfliction ​ ​ procedures. Test 3 Objective: ​ A total of 30 flights were flown in Test 3. The objective of Test 3 was to compare deconfliction and delivery results when the flight teams utilized pre-programmed flights to reach and return from the drop target versus manually flying to and from the drop target. The stakes were raised during this test because all 3 teams were to fly simultaneously (the two previous tests had only one team flying at a time). Test 3 Description: ​ Airboss A directed the first half of Test #3. Dronelink software was used to pre-program each of the 3 flight team’s 5 flights, for a total of 15 pre-programmed flights. Each team flew their set of flights from beginning to end simultaneously with the other 2 flight teams. The Dronelink pre-programmed flights were used to control the aircraft takeoff and its flight to and from the target. The pre-programmed flights paused automatically over the target to allow the RPIC and VO to coordinate the manual drop of the package. After the package was dropped, the pre-programmed flights resumed, which brought the drone back to the takeoff area where it hovered until the RPIC manually landed the drone. Airboss A had pre-programmed the Blue team’s flights to go out and back at 175 feet. The Green team’s operating altitude was 200 feet and the Yellow team’s operating altitude was 225 feet. This pre-programmed separation was intended to assist with deconfliction. Airboss B directed the second half of Test #3. DroneSense software was used to show the pilots the target location on a map. The pilots flew to and from the target manually at an altitude directed by Airboss B and the flight team handled the package drop manually. Return flights were also flown manually. DroneSense allowed the Airboss to see a dashboard showing a map of the real-time location of each drone and the camera view from all 3 drones simultaneously. This provided the Airboss B a central place to view the status of all the flights in addition to his physical proximity to the crews and radio communication. Airboss B manually deconflicted flights for this half of the test. Test 3 Location: ​ Each flight crew was assigned one set of drop locations. Yellow was assigned targets 1-1 through 1-5. Green was assigned 2-1 through 2-5. Blue was assigned 3-1 through 3-5. Each flight crew took off and landed in their respective landing areas in front of their canopies at the Launch 2 Area. Each team delivered a package to each of their 5 targets under both Airboss A using Dronelink and Airboss B using DroneSense.

Because all 3 crews were operating at the same time and flying all over the campus, this was a test of airspace deconfliction.

160 Newtown Road, Suite 302 | Virginia Beach, VA 23462 | 877-601-1860 | ​ droneup.com

20

Test 3 Resources: ​ 3 Inspire 2, 30 packages, 6 RPIC/LoadMaster/VOs, 15 drop locations, Dronelink software on all iPad controllers for Test 3A, DroneSense software on all iPad controllers for Test 3B, DroneSense software on a PC for Test 3B, communications radios. Test 3 Limitations: ​ For the purposes of test integrity flight crews performed the drop function using only the stock DJI Inspire 2 and supporting hardware, excluding the drop mechanism. All Inspire 2 batteries were fully charged at the start of each test. No other software than that being tested (Dronelink and DroneSense) were used in flight to complete the operation. The RPIC and VO maintained direct 2-way communication at all times during the test. Test 3 End State: ​ The test was complete when each flight crew attempted one delivery (successful or unsuccessful) at each of the 5 defined locations for each half of Test 3, for a total of 15 flights for each team.

160 Newtown Road, Suite 302 | Virginia Beach, VA 23462 | 877-601-1860 | ​ droneup.com

21

9.4 TEST 4: Hazard Scenarios – ​ Determine the performance impact of hazards ​ . Test 4 Objective: ​ The objective of Test 4 was to validate the adaptability of flight crews in dynamic environments with hazardous and dynamic obstacles. Test 4 Description: ​ This test challenged the flight crews to deliver in tight spots surrounded by obstacles that were well out of the line of sight, with possible command and control problems due to buildings. Also, dynamic (moving) hazards were introduced. Each flight crew was required to deliver a package to 3 locations. Pilots were not briefed on what hazards to expect and had to respond to and resolve each hazard in real-time to safely achieve the delivery objective. Test 4 Location: ​ Each team took off from and landed at Launch Area 2 marked by the red circle and flew to each hazardous target indicated by purple points ​ ​ on the map below (targets 4-1, 4-2, and 4-3).

Test 4 Resources: ​ 3 Inspire 2, ​ ​ 9 ​ ​ Packages, three flight crews (RPIC/LoadMaster/VO), one vehicle, communication radios.

Test 4 Limitations: ​ For the purposes of test integrity flight crews performed the drop function using only the stock DJI Inspire 2 and supporting hardware, excluding the drop mechanism. All

160 Newtown Road, Suite 302 | Virginia Beach, VA 23462 | 877-601-1860 | ​ droneup.com

22

Inspire 2 batteries were fully charged at the start of each test. No other software than DJI Go 4 was used in flight to complete the operation. The RPIC and VO maintained direct 2-way communication at all times during the test.

Test 4 End State: ​ The test was complete when each flight crew attempted one delivery (successful or unsuccessful) at each of the 3 defined locations for a total of 9 flights.

9.5 TEST 5: Cross-Platform Deconfliction

Test 5 Objective: ​ The objective of Test 5 was to identify command and control methods for the deconfliction and management of flight operations of multiple platforms in a confined operating environment. Test 5 Description: ​ One ​ DroneUp ​ flight crew and one Workhorse flight crew ​ ​ performed deliveries simultaneously to the same locations identified for test 5. As a part of their performance, ​ DroneUp ​ and Workhorse flight crews needed to coordinate with the Airboss to implement tactics to deconflict their operations. Test 5 Location: ​ Test 5 revisited the yellow and green points from Test 3.

Test 5 Resources: ​ DJI Inspire 2 drone, ​ DroneUp ​ Flight Crew, Workhorse drone, Workhorse Flight Crew, 10 packages, Communications Radios

160 Newtown Road, Suite 302 | Virginia Beach, VA 23462 | 877-601-1860 | ​ droneup.com

23

Test 5 Limitations: ​ For the purposes of test integrity the ​ DroneUp ​ flight crew performed the drop function using only the stock DJI Inspire 2 and supporting hardware, excluding the drop mechanism. All Inspire 2 batteries were fully charged at the start of the test. The Workhorse flight team utilized their standard controller and associated software. No other software was used in flight to complete the operation. The RPIC’s, LoadMasters, and VO’s all maintained direct 2-way communication with each other and the Air Boss during the test. Test 5 End State: ​ The test was complete when each flight crew attempted one delivery (successful or unsuccessful) at each of their 5 defined locations, for a total of 5 flights for each of the 2 teams. 9.6 TEST 6: Night Operations – ​ Determine the performance impact of night operations Test 6 Objective: ​ The objective of Test 6 was to understand the impact of night operations on drone deliveries and determine what, if any, different procedures must be implemented to ensure the safety and efficiency of drone deliveries at night. Test 6 Description: ​ Test 6 was performed identically to Test 2, but flight operations began after evening civil twilight. Test 6 Location: ​ Test 6 used the same locations identified for Test #2. Test 6 Resources: ​ 3 Inspire 2, 9 RPIC/Loadmaster/VO’s, red night lights for ground vision enhancement, anti-collision lights installed on the drones per FAA regulations, communications radios. Test 6 Limitations: ​ For the purposes of test integrity flight crews performed the drop function using only the stock DJI Inspire 2 and supporting hardware, excluding the drop mechanism. All Inspire 2 batteries were fully charged at the start of the test. No other software was used in flight to complete the operation. The RPIC and VO maintained direct 2-way communication at all times during the test.

Test 6 End State: ​ The test was complete when each flight crew attempted one delivery (successful or unsuccessful) at each of their 5 defined locations, for a total of 15 flights.

9.7 Proof of Concept 1: Thermal Imagery

Proof of Concept 1 determined how aerial thermography might be used to identify groups of persons who may be in violation of certain state-sponsored quarantine and distancing orders. An Inspire 1 drone outfitted with a FLIR XT thermal camera was used.

160 Newtown Road, Suite 302 | Virginia Beach, VA 23462 | 877-601-1860 | ​ droneup.com

24

Not only was the XT thermal camera able to identify the heat signatures of people at night, but drones themselves were discovered to emit a heat signature that could be readily detected by a pilot viewing the first-person view of what the XT camera was capturing. This is an actual photograph of the Inspire 1 controller viewing the XT camera output.

9.8 Proof of Concept 2: Aerial Broadcast Proof of Concept 2 determined how aerial broadcasting might be used to warn groups of people of potentially hazardous events without the need for public address system infrastructure, radios, or a functioning communications grid. A Mavic 2 Pro drone was outfitted with its broadcast speaker option and recorded messages were broadcast to the test teams gathered outdoors. The recordings were clearly audible by all attending with the drone at approximately 70 feet of altitude. 9.9 Proof of Concept 3: How loud in Decibels (dB) are Inspire 2 Drones? Proof of Concept 3 tested how loud the Inspire 2 drones were in delivery operations to ensure that they can operate safely in an emergency delivery environment without damaging the hearing of people nearby. Normal conversation is about 60 dB, a lawnmower is about 90 dB, and a loud rock concert is about 120 dB. The National Institute of Occupational Safety and Health (NIOSH) has established the Recommended Exposure Limit to avoid hearing damage as 85 dB averaged over an 8 hour period. A decibel meter was used to measure the decibels present when the Inspire 2 drone was at various altitudes. Here are the results of the test:

160 Newtown Road, Suite 302 | Virginia Beach, VA 23462 | 877-601-1860 | ​ droneup.com

25

Inspire 2 Drone Sound in Decibels Altitude Sound

Net Added

Description

(feet)

(dB)

(dB)

Ambient Noise (no drone)

0

42

0

Drone motors idling

0

62

20

Drone hovering

10

68

26

Drone hovering

25

62

20

Drone hovering

50

57

15

Drone hovering

100

50

8

Drone hovering

200

47

5

Drone hovering

250

45

3

Our flight altitudes generally ranged between 150 and 200 feet, so the decibel level of those deliveries was probably in the 47 to 50 dB range. The highest decibels anyone would hear were upon landing when the levels reached 68. That is slightly louder than a normal conversation and well below the NIOSH standards for preventing hearing loss. 9.10 Proof of Concept 4: Delivering a package to a Drone Mailbox Proof of Concept 4 demonstrated the AirBox Technologies solar-powered, secure, Internet-connected delivery receipt box. A signal was sent from the authorized drone operator to open the box, and the package was dropped into it using a controlled magnet at the end of a tether connected to a winch.

160 Newtown Road, Suite 302 | Virginia Beach, VA 23462 | 877-601-1860 | ​ droneup.com

26

Schedule of Operations

10

10.1 Day 1 Tuesday ​ DroneUp ​ Only 0900-1200 ​ DroneUp ​ Set Up 1200-1300 ​ DroneUp ​ Lunch

9:00-12:00PM 12:00-1:00PM 12:30-1:00PM 1:00-3:00PM 1:00-3:00PM 3:00-9:00PM 8:00-9:30AM 9:30-11:00AM 11:00-12:00PM 12:00-1:00PM 1:00-2:30PM 2:30-4:00PM 4:00-6:00PM 6:00-7:00PM 7:00-8:00PM 8:00-9:30PM 8:00-9:30AM 9:30-10:30AM 10:30-12:30PM 12:30-1:30PM 1:30-3:30PM 3:30-6:00PM

1230-1300 Pilot and VO Registration and Check-In 1300-1500 Partner Registration and Check-In

1300-1500 Pilot Briefings 1500-2100 Orientation Flights

10.2 Day 2 Wednesday

0800-0930 Day 2 Briefing 0930-1100 Test 1 ( ​ DroneUp ​ ) 1100-1200 Test 1 (Workhorse) 1300-1430 Test 2 ( ​ DroneUp ​ ) 1430-1600 Test 2 (Workhorse) 1600-1800 Test 3 ( ​ DroneUp ​ ) 1800 - 1900 Test 4 ( ​ DroneUp ​ ) 1200-1300 Lunch 1900 - 2000 Dinner 2100 -2300 Test 6 ( ​ DroneUp ​ )

10.3 Day 3 Thursday

0800-0930 Day 3 Briefing 0930-1030 Test 4 (Workhorse)

1030-1230 Test 5 ( ​ DroneUp ​ & Workhorse)

1230-1330 Lunch

1330-1530 Partner Demos

1530 -1800 Pack Out and Depart

160 Newtown Road, Suite 302 | Virginia Beach, VA 23462 | 877-601-1860 | ​ droneup.com

27

11 Testing Results

Forty-five people attended the event, half were Part 107 certified sUAS pilots. The weather was great for an outdoor drone exercise. There were a few periods of rain where flight operations were suspended. Day 3 had windy conditions with gusts over 20 mph.

April 7 (day): 80°, wind 6mph, humidity 40%, visibility 10 miles April 8 (day): 80°, wind 10mph, humidity 40%, visibility 10 miles April 8 (night): 65°, wind 5mph, humidity 75%, visibility 10 miles April 9: (morning): 65°, wind 10mph, humidity 70%, visibility 10 miles April 9: (afternoon): 75°, wind 15mph, humidity 45%, visibility 10 miles

Over 200 flights were flown and the 90 ​ DroneUp ​ team delivery flights were recorded in detail in the Inspire 2 log files analyzed by AirData software and by human Data Recorders. During the 90 delivery flights, one flight was aborted due to wind instability. The pilot during the one aborted flight dropped the payload near the takeoff area in a controlled drop. Of the 90 delivery flights, the ​ DroneUp ​ pilots successfully dropped the payload within a 10-foot diameter target 90 times for a 100% success rate. Successful demonstration tests included using drone thermal imagery to see humans and drones at night, using a drone as a public address system, measuring drone noise at different altitudes, and delivering a package to an automated drone package mailbox.

Test 1 Results

Test 1 was the baseline with 3 teams each delivering 5 packages for a total of 15 flights, all within VLOS. All 15 drops were within the 10-foot target circle. No demerits were scored. This chart depicts the distance, time, and speed results for the baseline. Test 1 Results will be compared to the more complicated tests to follow.

(All times in minutes)

Test 1

Test 1

Baseline

Results

Control

Difference

Number of Flights

15

15

0

Distance (total feet)

580.6

580.6

0

Time Ascending (Takeoff)

0.25

0.25

0

Time to Target

0.79

0.79

0

Time on Target

0.68

0.68

0

Time Returning

0.25

0.25

0

Time Descending (Landing)

0.31

0.31

0

160 Newtown Road, Suite 302 | Virginia Beach, VA 23462 | 877-601-1860 | ​ droneup.com

28

Total Elapsed Time

2.28

2.28

0

Speed to Target (mph)

4.8

4.8

0

Speed Returning

15.2

15.2

0

Battery Consumption

10.7%

10.7%

0.0%

This set of flights was easy for the pilots. At this short distance (round trip 581 feet on average) a crew could potentially deliver 4 packages on one set of batteries and 20 deliveries per hour.

Test 2 Results

Test 2 moved the flight crews to the top of the hill and moved the targets to Beyond Visual Line of Sight (BVLOS). The distance was increased by 81% resulting in a 1052 foot round trip (about three times the length of a typical city block). All 15 drops were within the 10-foot target circle. No demerits were scored. Here are the results compared to the baseline of Test 1.

(All times in minutes)

Test 2

Test 1

BVLOS

Results

Control

Difference

Number of Flights

15

15

0

Distance (total feet)

1052.4

580.6

471.8

Time Ascending (Takeoff)

0.49

0.25

0.24

Time to Target

0.91

0.79

0.12

Time on Target

1.75

0.68

1.07

Time Returning

0.47

0.25

0.22

Time Descending (Landing)

0.38

0.31

0.07

Total Elapsed Time

4.00

2.28

1.72

Speed to Target (mph)

7.5

4.8

2.7

Speed Returning

14.5

15.2

-0.7

Battery Consumption

19.3%

10.7%

8.6%

For Test 2, the increased distance of 81% combined with the introduction of a VO guiding the drop increased the elapsed time by 75%. The speed to and from the target actually increased, possibly due to practice. However, because the target was BVLOS, the time on target, as expected, more than doubled to almost 2 minutes. Under these conditions, a crew could potentially deliver 4 packages on a set of batteries and an estimated 13 to 15 deliveries per hour.

160 Newtown Road, Suite 302 | Virginia Beach, VA 23462 | 877-601-1860 | ​ droneup.com

29

Test 3A Results

Test 3A introduced pre-programmed flights to and from the target and remained BVLOS. This was the first test where all 3 crews flew at the same time. Deconfliction was accomplished in the pre-programmed flights by having each team fly at pre-set altitudes 25 feet vertically apart (175, 200, and 225). The distance for the 15 flights was increased on average by 54% over Test 1. All drops were within the 10-foot target circle. No demerits were scored. Here are the Test 3A (Pre-programmed) results compared to the baseline of Test 1.

(All times in minutes)

Test 3A

Test 1

Pre-Programmed

Results

Control

Difference

Number of Flights

15

15

0

Distance (total feet)

893.2

580.6

312.6

Time Ascending (Takeoff)

0.43

0.25

0.18

Time to Target

0.88

0.79

0.09

Time on Target

2.58

0.68

1.9

Time Returning

0.42

0.25

0.17

Time Descending (Landing)

0.79

0.31

0.48

Total Elapsed Time

5.10

2.28

2.82

Speed to Target (mph)

5.8

4.8

1

Speed Returning

12.8

15.2

-2.4

Battery Consumption

24.2%

10.7%

13.5%

Test 3A, on average, took over twice as long to complete than Test 1. The battery consumption increased as the speeds that were programmed to be conservative were slower than what the pilots flew manually. The elapsed time per flight went up 28% compared to test 2 even though the distance was shorter. Under these conditions, a crew could potentially deliver 3 packages on a set of batteries and an estimated 10 to 11 deliveries per hour.

Test 3B Results

Test 3B eliminated pre-programmed flights and remained BVLOS. The targets were the same as in Test 3A. This test used DroneSense software, which identified, and marked, the target on the flight controller map with a pin, which helped the pilots find it faster than a pure manual flight. DroneSense also provided the Airboss with a dashboard to view the location of each drone on a map and the video feed from each drone. This was the second test where all 3 crews flew at the same time. Deconfliction was accomplished by the Airboss setting flight

160 Newtown Road, Suite 302 | Virginia Beach, VA 23462 | 877-601-1860 | ​ droneup.com

30

Page 1 Page 2 Page 3 Page 4 Page 5 Page 6 Page 7 Page 8 Page 9 Page 10 Page 11 Page 12 Page 13 Page 14 Page 15 Page 16 Page 17 Page 18 Page 19 Page 20 Page 21 Page 22 Page 23 Page 24 Page 25 Page 26 Page 27 Page 28 Page 29 Page 30 Page 31 Page 32 Page 33 Page 34 Page 35 Page 36 Page 37 Page 38 Page 39 Page 40 Page 41 Page 42 Page 43 Page 44 Page 45 Page 46 Page 47 Page 48 Page 49 Page 50 Page 51 Page 52 Page 53 Page 54 Page 55 Page 56

www.droneup.com

Powered by