![[Screenshot 2024-06-13 at 5.27.02 PM.png]] I swung to far on the pendulum. [Driver Alert Dial(DAD)](https://publish.obsidian.md/typedbyhumans/Driver+Alert+Dial+aka+DAD) was on one side of the spectrum, while changing the color of the openpilot UI border to represent confidence level was on the other. DAD was to expressive and complex while changing the openpilot outer border color from green to yellow to red didn't communicate enough info to users on what to do next after seeing a color. **Skip to the design?** [Here](#Music) Go to Proof of Concept? [Here](#Driver+alert+cluster+POC) Figma File: [Here](https://www.figma.com/design/jsI0xQx1rW8KABSt2JObnV/DriverAlertCluster?node-id=0-1&t=FWlp7C4yk9umnCt1-1) Link to Discord post: [Here](https://discord.com/channels/469524606043160576/1257052290565673132) Github for DAC: [Repo](https://github.com/ugtthis/openpilot/tree/dac-ui) If you are running `replay` with this branch make sure to set your display scale to 100%. If scale is different the UI won't be the intended size. ![[Screenshot from 2024-06-29 09-55-34.png]] # Changing the border color ![[Screenshot 2024-06-27 at 10.25.28 PM.png]] This approach was to simple because even if you knew openpilot was uncertain about something, you did not really know why. There was no other warnings when yellow was going into red, so changes of confidence can appear abrupt. Being on yellow seems to be the confidence level openpilot is usually at, which can cause alert fatigue. Seeing a single static color to represent openpilot’s confidence state, doesn't communicate well whether it's not confident about gas, brake or steering. # Driver Alert Dial (DAD) ![[DAD-different-modes 1.png]] This approach was to expressive of what needs to happen. It is able to communicate much more of the spectrum of confidence and what to adjust, but comes with too much complexity to solve the current problem and a big assumption. # Alert ball not being helpful ![[DAD-UI-vid 1.mp4]] In the video above you'll see DAD start in a disengaged state(low opacity colors) then go yellow(medium alert) then red(high alert). The small circle thats moving is the alert ball. The image below gives a rough idea of what it means if the alert ball is in a certain position. GitHub link [here](https://github.com/ugtthis/openpilot/tree/dadpilot) for DAD. ![[DAD-key.png]] Having the alert ball stay in the middle is the sweet spot. That means it’s in low alert mode and openpilot will be pretty reliable, and need disengagement less than once every 10 min. At the time, I decided that the ball should show how much of an override you need to do to get the ball in the center. Your override will also affect how the ball moves. For example, when alert ball is on the far left side of the circle then you need to turn the steering wheel left more and openpilot will then bake in your applied torque and move the ball accordingly. As seen in the video above, the ball isn't quiet helpful right now. Even if I took away the alert ball, the alert dial becomes too simple and does the same thing as the first idea of changing between the 3 different colors. As I spent more time trying to develop and think through the DAD design I noticed more and more the hardest part of the design is going to be the alert balls movement. But even if I was able to get the alert ball movement to be more helpful then it is now there was a big baked in assumption that this UI tool should tell you exactly how to drive. # I changed The question that got me thinking that DAD may be more wrong than my future ideas, was a question asked by @bongbui321. Discord message [here](https://discord.com/channels/469524606043160576/1246611208509853777/1247211166657413221). ![[Screenshot 2024-06-13 at 5.52.36 PM.png]] **Should openpilot show users how they should drive?** This got me thinking about the whole purpose of the UI tool I was trying to create. "The goal of this challenge is to convey openpilot's driving confidence states to the user while they are engaged." This UI tool was **not** supposed to show you how to drive but, communicate the current confidence state openpilot is in. Maybe DAD tool will be brought back for another problem in the future, but the DAD tool didn't seem like the best solution for this context. Time to find a better design idea. # Music ![[Screenshot 2024-06-13 at 8.34.21 PM.png]] In most DAWs(Digital Audio Workstations) such as Logic, Ableton, Pro Tools there is something that looks like the above. It essentially tells you how loud a certain track is. It doesn't tell you how loud or quiet something should be, but it does tell you when something may "clip". This means the audio starts to sound distorted and if audio is clipping it could break certain speakers if you play the clipped audio with them. This is where my next design idea comes from. Just like the volume controls above, the UI tool shouldn't tell you how to exactly drive, or in this music example, tell you how loud or quiet a track should be in your mix. The UI tool should give expressive enough information for you to then make a decision off of it. This is where the Driver Alert Cluster comes in # Attempt #2 - DAC ![[Group 6(1).png]]The Driver Alert Cluster(DAC) shows 3 status bars that represents steering, gas and brake. Each status bar represents how alert the driver should be for each respective label. If gas is cyan, it is low alert. Most likely you are on a straight road for miles and openpilot calculates that a quick disengagement is unlikely. If you see red for brake that would mean you could be going fast and a stopped car is approaching. Red is high alert and points out that the user should be focused on driving to see what needs to be done for braking or whatever the respective label is. As mentioned in the music section, the DAWs have volume controls for each track in which the producer/mixer decides how loud something should be. You could say the DAC has 3 tracks that show the volume just like a DAW does. Red means, be extra careful, and it is likely that negative consequences could happen **unless** **adjusted**. Or for this case, openpilot is expected to not work, and will likely need disengagement **within seconds**. ## Different UI states ![[Group 7.png]] **Cyan - Low Alert** Pretty reliable, needs disengagement less than once every 10 min **Yellow - Medium Alert** May or may not work, needs disengagement every few minutes or more **Red - High Alert** Expected to not work, will likely need disengagement within seconds **Muted Colors - Disabled** openpilot is not currently engaged # Driver Alert Cluster POC ![[ui-screenshot.png]] Above image shows the DAC being used with a demo route on replay. It's showing the steering bar in high alert suggesting that disengagement may happen because of the current environment. Purpose of DAC is to communicate what openpilot is thinking, so users can better anticipate overrides, and prevent the bigger abrupt "take control immediately" alerts from happening. Less abrupt, **unexpected** big alerts, the more chill the drive becomes. Want to checkout the repo for the Driver Alert Cluster. Click [here](https://github.com/ugtthis/openpilot/tree/dac-ui) ### During `replay` - Experimental mode ![[dac-replay.mp4]] Above shows a glimpse of the DAC in action. The car is taking an off ramp and the steering bar is going into the red to tell the user "Hey! Something is about to happen, please check steering cause I don't know if I can drive much longer". For this case I would consider the steering bar did it's job successfully, because it was able to go into the red before the alert came. ### Going off the freeway - chill mode ![[offramp-dac.mp4]] In this clip there is a curve with than a quickly approaching red light. This was not on Experimental mode like the last clip. In this drive, I noticed the brake bar went high alert right when it saw the red light after the curve, which was great because the car was not slowing down since there was no lead car and it was on chill mode. The brake bar reflected how I felt what should happen. ### Small roads - Experimental mode ![[2lane-dac.mp4]] Driving on a 2 lane road with stop signs in a neighborhood. One thing I thought was cool, was in the beginning of this clip you will see a girl with a longboard on the right side of the road and a white car in the left lane. The steering bar went high alert, which was a great sign to see because it also reflected how I felt in that scenario. My thoughts were, it's a small road, there is a moving car to my left and a moving human approaching to my right. It would suck to hit one of them therefore I should be extra careful here. # Things I'm still thinking about ### Color changes are jumpy The way the circles go from low alert to high alert can be jumpy. This needs to be smoothened out where each color change really represents the confidence of openpilot for the specific bar ### Braking disengagement predictions = not that helpful yet More helpful than gas but less helpful than steering. There were multiple moments on the freeway when I was using [dac-ui](https://github.com/ugtthis/openpilot/tree/dac-ui) branch where I expected it to tell me it's not confident Ex. I was on the freeway and I couldn't see whether the cars in front of me were going to come to a complete stop or keep going. openpilot was still accelerating and not slowing down so I took over. A successful interaction would be the brake bar would start hitting the red to let me know it is unsure whether the cars in front won't slow down rapidly. ### Is the gas bar needed? I used the DAC UI tool the other day driving it on curvy roads, city roads, freeways, and not once did I see the gas bar hit high alert. Right now the logic in the code is just asking whether the disengage prediction is above a certain number. Steering, brake, and gas are using the same logic to determine what alert level it will be in. The only time I saw the gas bar move was when it seemed like my car was in a RPM that it shouldn't be in. If I recall correctly, my car was making a weird sound like it wanted to be in a different gear while engaged so then I disengaged and took over. Ideally, the gas bar should go into medium or high alert whenever it notices that maybe this car should be going faster because all the other cars nearby are going at a faster speed. Ex. I was at a traffic light and the car was acting slower than usual to move from a stopped position when the light turned green. It had a lead car, and and another car to its left. When the car in front and left took off because of the green light I then took over, since openpilot was taking longer than usual to proceed forward. I was on chill mode which may have played a factor. ### Adding a DM bar? I've been thinking about how to communicate DM is more alive. I went on a roadtrip the other week with my sister. She was asking about the comma device I had. I started talking about how the device will get mad at me If I'm distracted. I then tried to show her that a alert will pop up, and for some reason it was not happening. Even if it was working, but DM decided I was not distracted enough, there was no way in knowing because all you see is a icon moving suggesting DM is on. Yes, the current DMoji icon does show that it's tracking your movement, but there is not really any meaning behind the icon movement other than DMoji is on. It is helpful for being transparent that the camera is watching you, but not helpful whether it can actually communicate and tell the difference between whether the driver is being focused or distracted, and to what degree. If I think back to my story with my sister, showing her a bar that changes color depending on how distracted I am can better illustrate the device does recognize you are moving, but more importantly that it has an opinion about that movement. ### How many bars is too many bars? If a DM bar is added thats 4 bars on the UI screen. I'm not confident yet about the answer to this. Maybe the DM bar will look different. Or maybe DMoji gets a facelift and stays a circle and we are able make it change colors doing what the DAC bars do, but slightly different. If it stays a circle, then we can put this DMoji in the bottom right of the screen since the Map button is not there anymore. Or maybe DMoji icon should stay as is, and the system improves so that it can better show alerts when you try to demonstrate to other you are distracted, and that the device will notice and remind you to focus. ### Updates, posting feedback , and comments? For more progress or to report feedback about the DAC, I will be posting in `#current-projects` in Discord. Link to Discord post is [here](https://discord.com/channels/469524606043160576/1257052290565673132). This will be the main channel where I will be posting video examples of test drive, commenting about ideas or getting feedback from others. # Conclusion Not much to write here. Not much has changed since the conclusion of [DAD post](https://publish.obsidian.md/typedbyhumans/Driver+Alert+Dial+aka+DAD#Conclusion) so refer there for the conclusion. But since your here I'll end with my favorite John Carmack quote. "You can't know everything, but you can learn anything." Thanks for readin! --- Github for DAC: [Repo](https://github.com/ugtthis/openpilot/tree/dac-ui) Figma File: [Here](https://www.figma.com/design/jsI0xQx1rW8KABSt2JObnV/DriverAlertCluster?node-id=0-1&t=FWlp7C4yk9umnCt1-1)