Pros and Cons

Pro

1. The different tests were very good. There were many test elements in there that really formed meaningful evaluation criteria. It could be seen that the setup was done by people who had experience with mobile robots and their capabilities as well as limits.
2. An other very big pro point was the high number of professional teams. It was highly interesting to see their performance.
3. The overall organization was good and one got the feeling that there was the wish to support the teams as much as possible.

Con

1. Some more "hard" information about the background of the event (motivation and goals) would have been nice. But I guess that is simply owned to the military as main organizer.
2. Also, some summary of the findings of the event would have been very desirable from the academic as well as the practitioner's perspective. The results could have been anonymized, respectively presented in a general manner. As mentioned above, I think the scenarios included several interesting test elements, which on their own would justify a publication. But maybe the goal is anyway to still get a scientific publication out (which would simply render this "contra" meaningless :) ).

Remarks

1. In general, a nice event that I found very helpful to get some first hand experience about the current state of the field. As mentioned before, the high number of professional teams gave an interesting touch. But also the systems from the academic participants were interesting.

Pro

1. The test beds (urban and dirt site) were ideal from a tactical point of view: it demonstrated the complexity of real nature.
2. It was a good chance to demonstrate that full autonomy will not be a real option in the next few years.
3. Both sites provided the opportunity to demonstrate mobility and reconnaissance performance.

Con

1. The participants must get feedback on what the referees liked about their own robots and what they dis-liked, i.e. what was missing. Feedback is fundamental for evolution - not only in nature but in industry as well.
2. The demonstration of robots "downtown" on tuesday was dissatisfying. No single test site, no schedule, ... Since the companies and institutes spent an awful lot of money to support the ELROB initiative ELROB shall provide a good platform for the demonstration of the individual capabilities that were beyond the scope of ELROB.
3. Handling and remote control of robots is and will be a major issue in the future. Therefore the man machine interface, i. e. the controls and displays, shall be presented to the visitors and observers as well.
4. The dislocation of the event into non urban trial on the hill, the urban trial and the exhibition area far away from anything else was poor.
5. The presence of press and TV media was much too high. Some colleagues were only busy in giving interviews, especially those, who did not participate in the trials with their vehicles.
6. The contact between the teams should be intensified, e.g. by a social event on the first evening.

Remarks

Pro

1. It was great to have such a well prepared event on robotics in Europe !

Con

1. It was unsatisfactory, that no feedback about evaluation results was provided to the teams (to each team only about their own robot, not about other robots).
2. Also some kind (of much more limited information) about strengths and weaknesses of the different robots could be provided to the public.

Remarks

Pro

1. It is highly appreciated that the German Federal Armed Forces (!) have organized this trial in order to present an overview of the European state-of-the-art UGVs.
2. Hammelburg provides an ideal environment for such an event.
3. Commitment of the organizers
4. Effective barriers
5. Facility for practicing, even for bigger vehicles
6. Good organization
7. Excellent support provided by both, military and civilian staff, irrespective of rank and status.

Con

1. Some turns on the course were too narrow for big vehicles.
2. The responsibilities of the official observers were not clear.
3. The appraisal of results has not been published, yet.
4. Visitors have no information concerning the workload of the operators. Possibly a video display could help.

Remarks

1. The exhibition and trials provide a good and compact overview of the German/European state of the art UGV technology not only for the organizers, but also for the industry.

Pro

1. The event is a good opportunity to meet other people working on the same field and of course to evaluate the performances of our machine in real conditions.
2. We realy appreciated the help of the Elrob staff since we had some trouble with our robot !

Con

1. The staff of Elrob is asking too many documents and information before the trial. In the laboratory, we already have too many papers and forms to feel, maybe you should ask less paperworks.

Remarks

1. At the university, the design of the robot started 9 month before Elrob, at the beginning, Elrob was presented as a kind of contest. The fact it was finally a trial is a good thing, the atmosphere of the event was great. But on the other hand, when we went back in France lot of people (also our supervisors) asked us about our results. Maybe, having a rank, a kind of trophy, a certificate or a statistical sheet about the performances of the robot may be nice.

Pro

1. The ELROB was a great event. One had the chance to test the developed robot equipment outside the lab. Real test are completely different from test in the laboratory. The test site is a great place for exciting and challenging robot runs.
2. The advanced technology showed during the exhibition and the trials produced nice starting points for socializing with representatives from industry and research and resulted in a lot of fruitful discussions.

Con

1. The test was designed to present reliable robot control only. Issues of situation awereness haven#t been really tested, since there was a line that had to be followed and the teams where allowed to see the course in advance (and to take measurements at critical points, i.e., at the bridge or at the small alley). It would have been interesting to see the teams performing steering their robot if the operator had only an aerial photo before the run.
2. Other state-of-the-art robotic techniques, i.e., mapping, have not been tested, therefore the urban trial was not able to reveal the differences between scientific papers, that are nowadays presented, and the current development. Some commentator used this to advertise their products with things they aren't barely able to show. This is also true for the Non-Urban scenario.
3. Last but not least one have to mention that the ELROB event has similar events in the U.S. There is even much more interest in Urban Search (and Rescue), resulting in key words like US&R, CRASAR, Response Robot Evaluation Exercises, etc. pp...

Remarks

Pro

1. Very good organised.
2. Precise schedules and instructions.
3. Good monitoring of radio communications.
4. Fair arrangement of the parcours.

Con

1. Disqualified teams (Diehl/Robowatch) should remain disqualified or it should be stated to the audience when they are allowed to participate later in the trial.
2. In the Non-Urban trial it would have been desirable to have a greater range of difficulties in the sand and field section, because they were mainly designed for greater vehicles.

Remarks

1. You had many opportunities to see our teleMAX in use and to judge how simple and effective the system can be controlled.
2. I would be glad about any feedback, that you can give us about our system. Where are in your opinion the strengthen and weaken and if possible give us a comparison to our competitors (if necessary anonymously)
3. We would be glad to participate in the next ELROB and to meet a new challenge.

Pro

1. Very Good Location.
2. Impeccable organization.
3. Good level of attendees.
4. Excellent training area and Technical area.
5. Very good pictures on website and very good video coverage of trials (at least the ones I saw).

Con

1. The Barbecue (food was finished after 45 minutes)! The closing speech should have been made a day earlier (everybody had left by the end).
2. Exhibition area should have been more concentrated, the location we had was too remote from the rest of the booths and people were a bit confused by the separate booths (it probably would have been better to have a bigger group of tents).
3. Too many people around the robot during the trials generating problems with wireless signal.

Remarks

1. We should try to get all the responsibles for the Soldier modernization programs in Europe Involved (not just the robotics programs but the various FIST, Normans, Soldado do Futuro, Soldato del Futuro, FELIN and all the rest).
2. As training course it would be good also to have a section of Bonnland available as well (to test the performance in urban scenario as well as open scenario).
3. It might make sense to have a day in which the teams present their robots and rationale of design to the press (either before the trials or in a separate event) to avoid robots doing things that spectators do not understand or look very slow while in reality they are performing very well on autonomous behaviour.
4. We should also incentivate multi-robot performance, either in the trials or in some other application (marsupialism, UAV-UGV cooperation or similar).
5. We must push more on the press releases before and after the event and maybe start uploading pictures from the event at the end of every day (even if it is very few pictures this will keep attention up).
6. A press conference with all team leaders could also be good so journalists can ask questions after seeing the trials.
7. Also a Q&A moment at the end of the trial could be good so the team leader can answer questions from the public.
8. We should add a "spectator robot driving test" event in which spectators can try the robots themselves (in a controlled environment) in order to generate empathy with the systems and show that the robots are for real and not just some geek kind of thing. For this part of the work it would be good to have a small test area prepared for this purpose with some simple obstacles and different types of terrain.
9. I think we could also add sand somewhere in the trials (both urban and non-urban) as well as deep water (both with bypass possibilities).
10. I agree with the idea of eliminating the white marks on the floor but we might leave some marks for the bypass areas only in order to avoid robots ending in areas where they definitely are not capable of going.

Pro

1. We thought the event was very well co-ordinated, we had the technical support, hard weather proof garage, space and power necessary to allow us to set up and prepare our RCVS.
2. Security was good and we did not loose any equipment.
3. We liked the concept of the RED runs but for less capable vehicles even some of these RED RUNS were quite daunting.
4. We liked the figure of eight circuit below the high control point to give the best chance of radio communications, we also liked the idea of doing the radio range test after the trial run. Perhaps in a future ELROB there may be a new event dedicated to RCV comm's were the task is to see how far the RCV can go in adverse condition before control/video is lost?
5. We would just like to say we thought ELROB No1 was very well run and you and your team should be congratulated on the event.

Con / Remarks

1. While we fully understand that each team has to undertake each scenario some of the spectators advised they would have liked to wander between scenarios on a single day to get a general overview of capabilities.
2. As a competitor we would have liked the opportunity to attempt the set objectives in any order. That would have allowed us to attempt the easier low risk test areas before committing ourselves to the more risky tests especially those where there was a risk of damaging the RCV and thus failing to complete other aspects of the course.
3. For example we put Black Max through the mud pool in the morning but got it stuck. If we had had the opportunity at the end of the LONGCROSS run we would have attempted running the mud pool with LONGCROSS, safe with the knowledge that she had completed all the other less onerous tasks.
4. The option of selecting or not attempting set test piece may also have allowed those organisation that pulled out of some events to remain but only undertaking those test zones they felt comfortable with.
5. Having access to Telelifters, Military all terrain fork lifts etc to allow rapid recovery of bogged, failed RCVs would also have allowed us to take more calculated risks knowing that should we get bogged it would be but a few minutes to attach a strap and get the RCV unbogged ready to continue.
6. A few more obstacles, stairs, kerbs or slopes etc in the lower spectator demonstration area would have afforded us the opportunity to demonstrate the capabilities of the RCV in a non competitive arena.
7. The test zones were realistic but if only mobility was being tested then the option to control the RCV in line of sight mode would have aided those with limit real time non-line of sight operations.

Pro

1. organisation at Bonnland (e.g. compliance with the time schedule)
2. infrastructure at Bonnland (e.g. garages for the robots)
3. medial attraction to the event
4. quantity of decision-makers attending
5. strengths and weaknesses of the teams were visible by the trial
6. experienced environment
7. monitoring and compliance of radio frequencies during the mission
8. inspection of the track on monday (equal knowledge for all teams)
9. display-session on Tuesday

Con

1. imprecise description of the scenarios in the run-up to the trial (compared to the Grand Challenge)
2. no (satisfactory) possibility to present the grafical user interface to the spectators during the mission to demonstrate alternative solution of robot control
3. catering at OHG Hammelburg (mentioned by the PhD students: location at Bonland, selection of menues and not enough barbecue for the teams, which arrived late at the event due to trial preparation)

Remarks

1. partly unrealistic scenarios (e.g. white lines on the ground marking the track, operator station inside the village which has to be explored)
2. a possibility to demonstrate autonomy, would be good to better integrate research
3. a final feedback from the organisers and judges about the performance of the participants would be helpfull for further research of the teams

Pro

1. It was very nice for us to be able to take part in the event ELROB2006!
2. Our project will grow dramatically from the input we got douring the event in Hammelburg.
3. The infrastructure was very good!
4. Also the organization of the event!

Con

Remarks

1. If some funding would be possible next time, that would be great.
2. The aim of ELROB is not clear to our team and sponsors. It would be nice to know about the aim of the next elrob.
3. A party (get together) organized after the event would be very desirable.

Pro

1. Richard and i were very impressed with the organisation and enjoyed the relaxed atmosphere. Well done Elrob !!

Con

Remarks

1. Exhibition area needs to be more central so that essential movement of robots etc can be done in minimum time.Also more people would visit the exhibition area.
2. Signs would help to speed movement to events etc.
3. Large screen display coupled to drive cameras on each robot similar to uav events ,this is easy to organise and is used often in air display type exhibitions.The connection could be made when each team set up monitors at the start position.

Pro

1. Website communication all along the preparation steps
2. Organisers availability (fast answers to e-mails)
3. Bonnland area, that is perfect for urban and non urban trials
4. Welcome was excellent
5. Host by Hammelburg facilities was perfect : barbecue, cantina, etc ...

Con

1. Nothing

Remarks

1. To establish a placing of the concurrents
2. To add a "fixed obstacles" test (e.g 10cm step, a stair, a bump ...) with points relative to the difficulty of the obstacle
3. To fix a localization kit (GPS+radio) on the on-going robot, in order to show its localisation on the large screens for the audience.
4. to award the best competitor at the end of the trials, accordind different categories : urban, non urban, professional, university ...

Pro

1. Excellent event, professionally organised with good humour.
2. Website was used effectively to communicate throughout, and the post event pics were a significant benefit.

Con

1. Not much really. We should have sent one more person.
2. The only disappointing thing was that Remotec missed out largely n the Press Day, which we had not registered as happening. Was it publicised?

Remarks

1. We will be keen to take part again. We may well enter more products next time.

Pro

1. Die Organisation war im Großen und Ganzen OK (bis auf Ausnahmen, siehe unten ).
2. Die Unterbringung unseres Teams, Garage, in Bonnland (es waren aber nicht alle Teams mit ihren "Garagen" zufrieden).
3. Ein getrenntes Übungsgelände für große Fahrzeuge.
4. Verpflegung

Con

1. Anmeldeprocedere am Eingang Bonnland.
2. Es waren nicht alle Zuständigkeiten im Vorhinein abgeklärt und die Zuständigen waren teilweise schlecht erreichbar.
3. Der W-LAN Accespoint der T-COM in Bonnland war störend.
4. Frequenzmanagement am Dienstag und Donnerstag.
5. Wenn bestimmte Details, z.B. Antennenhöhen und Sendeleistungen, vorher genau festgelegt werden, sollte dies dann auch für alle Teilnehmer gelten und von den Organistoren überwacht werden.
6. Das Übungsgelände war, unter dem Off-Road Aspekt, wenig herausfordernd.
7. Das Auswerteteam war am Donnerstag teilweise erst nach den ersten Starts vor Ort.
8. Am Outdoor-Kurs gab es keine Videoübertragung für die Zuschauer.

Remarks

Pro

1. The event has been well organized.
2. The place was quite interesting and its conditions were good.
3. The Urban scenario, the competition in which IdMind participated, offered an interesting test-bed for the robots.

Con

Remarks

1. I noticed that in the Urban Competition some teams were using signal repeaters, for communications, placed in some other points then the one where were the operator and the main antenna. And that was based on previous knowledge of the critical places where the robots could face problems of communications with the operator. I do not know if that makes sense if we think in a realistic situation. I think that the antennas should be positioned only in one place.
2. For future events, in the Urban Competition the robots could also be requested to measure ambient conditions (e.g. temperature, humidity and presence of nocive gases).
3. There should be also a display session where teams could present their robots and their technology to the other teams.

Pro

1. Both technic and sales were present but isolated from each other.
2. Daily scheduling visible, events well coordinated but flexibility maintained.
3. The event in general very well organised.
4. The clear grouping into specialities . Urban / Nonurban / EOD.
5. The team areas of operation were well delegated. (large enough and situated that each team could perform as necessary).
6. The atmosphere. Competitive yet still friendly. Each team against the course, not (directly) each other.

Con

1. Sales area too distant - The food and facilities were located very far away from the AO's. Probably due to the security measures but still a hinderance.
2. No possibility of conducting/presenting autonomous navigation/behavior.
3. Channel usage monitoring could have been tighter; it took too long to detect interference.

Remarks