County of Los Angeles Vsap 2.0 Consultant's Accessibility Testing Report Attachment C

FCMG: Accessibility Test Report Testing Logs Attachment C - “Accessibility Testing Logs” Accessibility Tests – Primary and General Election FCMG tested the VSAP to ensure that it satisfied the requirements for voter interface and interactions in accordance with applicable statutes and standards. Characteristics of the system such as ballot appearance, languages, input methods and feedback were examined and noted in the CVSS Requirements VSAP Accessibility Testing References Spreadsheet (Attachment D). It should be noted that the following log is only a sample of the testing that was performed and does not include every voting session that was performed. Observation/ Screen Audio Physical task Voting Session #1 Began voting session after reopening the polls. I quit voting session and it ejected ballot. It took me back to language screen. When I went to continue it did not prompt me to insert ballot. Restarted test by Welcome to the June 01, 2016, rebooting. PRESIDENTIAL PRIMARY ELECTION 2016 Touch the circle to start (Multiple languages scrolling through under English text) For audio, please put on the headphones. Put on headphones Sound level was very low in headphones. Pushed + Volume one Went to “Choose language” screen Pressed volume 3 times before Last update: 2019‐12‐24 Status: Final Version: 1.4 Page 1 of 60 FCMG: Accessibility Test Report Testing Logs Observation/ Screen Audio Physical task (1) time with multiple languages listed below in I could hear instructions. boxes. “….Choose your language on the touch screen or press the round button on the controller to continue in. English. To hear the next language press the down arrow” Pressed round button to English is highlighted. “You have chosen English. select English To confirm English, press the next button on the touch screen or press the right arrow on the controller. To hear the next language press the down arrow.” Pushed the right arrow Insert your ballot “Insert your ballot, on the controller Insert the paper ballot into the tray to Insert your ballot into the paper your right tray to your right On your ballot one corner has been cut off. Hold your ballot with cut corner at the top right and insert it into the paper tray until it gets pulled inside Voting Session #2 To continue press the right arrow Pushed the right arrow Insert your ballot “Insert your ballot. on the controller Insert the paper ballot into the tray to Insert your ballot into the paper your right tray to your right On your ballot one corner has been cut off. Last update: 2019‐12‐24 Status: Final Version: 1.4 Page 2 of 60 FCMG: Accessibility Test Report Testing Logs Observation/ Screen Audio Physical task Hold your ballot with cut corner at the top right and insert it into the paper tray until it gets pulled inside Voting Session #3 Printer Paper Jam Printer Paper Jam Contact an election worker to help fix Contact an election worker to the printer help fix the printer Inserted ballot Loading ballot In some instances, audio would not give any message. Please wait a minute When in other ballot test, it would state “Loading your ” and no further audio Ballot loaded “Your ballot has been” No further audio (incomplete message) Your ballot: No further audio (incomplete message) Voting Session #4 How to use the controller. Let’s go through how to use the controller Graphic of controller How to use the controller Hold the controller so the cord screen come after faces away from you. selection of English There are 4 arrows in the language middle with a round button on the right. Last update: 2019‐12‐24 Status: Final Version: 1.4 Page 3 of 60 FCMG: Accessibility Test Report Testing Logs Observation/ Screen Audio Physical task Use these buttons to navigate through the ballot At the beginning of each contest you can press the left arrow to go back to the previous page. At the candidate list you can press the left arrow to repeat the previous instructions. To skip these instructions at any time, press the right arrow Use these You can use buttons to navigate through the contests At the At candidate list you can press the left arrow to repeat the previous instructions The Help button is located on top left The Audio volume buttons are on the top right. The Playback rate buttons are on the bottom right The Pause Play button is on the bottom left To go back to the previous page, press the left arrow. Last update: 2019‐12‐24 Status: Final Version: 1.4 Page 4 of 60 FCMG: Accessibility Test Report Testing Logs Observation/ Screen Audio Physical task Let’s get started “Let’s get started” I want to start voting No further message. I have a Poll Pass to scan This happened multiple times. How to use the controller Let’s go through how to use the controller Graphic of controller Hold the controller so the cord faces away from you. There are 4 arrows in the middle with a round button on the right. Use these buttons to navigate through the ballot At the beginning of each contest you can press the left arrow to go back to the previous page. At the candidate list you can press the left arrow to repeat the previous instructions. To skip these instructions at any time, press the right arrow Use thes You can use buttons to navigate through the contests At the At candidate list you can press the left arrow to repeat the Last update: 2019‐12‐24 Status: Final Version: 1.4 Page 5 of 60 FCMG: Accessibility Test Report Testing Logs Observation/ Screen Audio Physical task previous instructions The Help button is located on top left The Audio volume buttons are on the top right. The Playback rate buttons are on the bottom right The Pause Play button is on the bottom left To go back to the previous page, press the left arrow. Turn screen on/off To turn screen off and show your selections To keep the screen on and show your selections press right arrow Pressed down arrow Let’s get started Let's get started I want to start voting You have 2 options I have a Poll Pass to scan Start voting or use the poll pass To hear the first option press the down arrow Voting Session #5 Let’s get started Let's get started Pressed down arrow I want to start voting You have 2 options I have a Poll Pass to scan Start voting or use the poll pass To hear the first option press the down arrow Pressed down arrow, Let’s get started No pressed round button, Last update: 2019‐12‐24 Status: Final Version: 1.4 Page 6 of 60 FCMG: Accessibility Test Report Testing Logs Observation/ Screen Audio Physical task pressed other buttons I want to start voting CVSS Reference: When touch screen was I have a Poll Pass to scan 3.2.8.b. selected then selection would occur and corresponding audio No selections were made would play After about a minute, the screen showed that a selection was highlighted “I have a Poll Pass to scan” and Audio began again. Pressed down arrow I want to start voting box is highlighted To start voting and navigate in blue through the ballot press the round button To hear the next option I have a Poll Pass to scan box in black press the down arrow. Pressed the down arrow I have a Poll Pass to scan is If you have a poll pass push the highlighted in blue round button. To hear the instructions. Press the down arrow. Pressed the down arrow Let’s get started You have 2 options Start voting or use the poll pass To hear the first option press the down arrow. To hear the first option press Last update: 2019‐12‐24 Status: Final Version: 1.4 Page 7 of 60 FCMG: Accessibility Test Report Testing Logs Observation/ Screen Audio Physical task the down arrow Went back to start voting “I want to start voting” is shown in Let’s get started and navigate through the black box with white letters. You have 2 options ballot Start voting or use the poll pass To start voting and navigate through the ballot press the round button. To hear the next option press the down arrow. Voting Session #6 “I want to start voting” is shown in Let’s get started black box with white letters. Went back to start voting You have 2 options and navigate through the Start voting or use a poll pass. ballot To hear the first option, press the down arrow. (After pressing the down arrow) To start voting and navigate through the ballot press the round button. To hear the next option, press the down arrow. (After pressing the down arrow) If you have a poll pass press Last update: 2019‐12‐24 Status: Final Version: 1.4 Page 8 of 60 FCMG: Accessibility Test Report Testing Logs Observation/ Screen Audio Physical task the round button. To hear the instructions press the down arrow. (After pressing the down arrow) Let’s get started You have 2 options Start voting or use a poll pass. To hear the first option, press the down arrow. (After pressing the down arrow) To start voting and navigate through the ballot press the round button. You have chosen to start voting and navigate through the ballot. To confirm your selection, press the right arrow. Pressed round button Make your selections “Make your selections There are 15 contests on today’s There are 15 contests on ballot. ballots on today’s ballot, When you reach the end, you will be When you reach the end you able to review your selections will be able to review all of your selections.” Back Next To go to the first contest, press Last update: 2019‐12‐24 Status: Final Version: 1.4 Page 9 of 60 FCMG: Accessibility Test Report Testing Logs Observation/ Screen Audio Physical task the right arrow. Pressed the right arrow Non-Partisan Offices No audio for text shown on screen. All voters, regardless of the party preference they disclosed upon registration, or refusal to disclose a party preference, may vote for….” Voting Session #7 Non-Partisan Offices “Make your selections All voters, regardless of the party There are 15 contests on preference they disclosed upon ballots on today’s ballot, registration, or refusal to disclose a When you reach the end you party preference, may vote for….” will be able to review all of your selections.” Voting Session #8 Non-Partisan Offices To continue voting press the right arrow. All voters, regardless of the party preference they disclosed upon To go back to the previous registration, or refusal to disclose a page, press the left arrow. party preference, may vote for….” Contests 1 of 15 City/Local State Senator 25th District There are 6 options Vote for less than be greater than 1 be greater than / be Last update: 2019‐12‐24 Status: Final Version: 1.4 Page 10 of 60 FCMG: Accessibility Test Report Testing Logs Observation/ Screen Audio Physical task greater than You have not made any selections on this contest Voting Session #9 Non-Partisan Offices Contests Pressed left arrow to go All voters, regardless of the party 1 of 15 to previous screen preference they disclosed upon City/Local registration, or refusal to disclose a party preference, may vote for…. State Senator 25th District There are 6 options Vote for less than be greater than 1 be greater than / be greater than You have not made any selections on this contest Pressed round button CHRIS CHAHINIAN “CHRIS CHAHINIAN Party Preference: Democratic Party Preference: Small Business Owner Small Business Owner” (shown as selected in Black box with white font) Your selection is Chris Chahinian. You have the maximum number of selections. Pressed right arrow CITY/LOCAL CONTEST 2/15 Contest 2 of 15 MEMBER OF THE STATE City/Local ASSEMBLY, 43rd District Pressed right arrow Member of the State Assembly, again to skip voting (No Vote for ONE 43rd District message that no Selections left:1 There are 9 options selection had been made Last update: 2019‐12‐24 Status: Final Version: 1.4 Page 11 of 60 FCMG: Accessibility Test Report Testing Logs Observation/ Screen Audio Physical task when going to next race) (List of candidates) Vote for less than be greater than 1 be greater than / be greater than You have not made any selections on this contest Pressed right arrow CITY/LOCAL CONTEST 3/15 Contest 3 of 15 UNITED STATES City/Local REPRESENTATIVE, 30th District, Vote UNITED STATES for ONE REPRESENTATIVE, 30th Selections left:1 District, (List of candidates) There are 7 options Vote for less than be greater than 1 be greater than / be greater than You have not made any selections on this contest Pressed down arrow PATREA PATRICK Audio read Candidates names Party Preference: Democratic and information except for what Pressed round button to was listed after colons after 3rd candidate Documentary Producer/Author Party Preference. Pressed right arrow COUNTY Contest 4/15 Contest 4 of 15 SUPERVISOR 5th DISTRICT City/Local COUNTY SUPERVISOR 5TH District There are 8 options Vote for less than be greater than 1 be greater than / be greater than You have not made any Last update: 2019‐12‐24 Status: Final Version: 1.4 Page 12 of 60 FCMG: Accessibility Test Report Testing Logs Observation/ Screen Audio Physical task selections on this contest Pressed down arrow MITCHEL ENGLANDER Audio read Candidates names and information Pressed round button to Councilmember 2nd candidate “MITCHEL ENGLANDER Councilmember” Pressed right arrow COUNTY Contest 5/15 Contest 5 of 15 DISTRICT ATTORNEY City/Local Vote for ONE Selections left: DISTRICT ATTORNEY 1 There are 1 option JACKIE LACEY Vote for less than be greater than 1 be greater than / be Los Angeles County greater than You have not made any selections on this contest Write-In Candidate Write in candidate is not Only write in a candidate if they are not counted in the number of listed above. available options. CVSS Reference 3.2.5.e.ii. Attempted to vote for Write in candidate. Nothing was read in Audio to state there was a Write In option. Using arrow keys and round button could not get to the Write in selection. It would only cycle from selection of Jackie Lacey as candidate to County Contest 5 of 15. Last update: 2019‐12‐24 Status: Final Version: 1.4 Page 13 of 60 FCMG: Accessibility Test Report Testing Logs Observation/ Screen Audio Physical task I was able to select Write in option by making selection on touch screen. (Note: this would not be possible for blind voter to accomplish without assistance). Voting Session #10 Type candidate name “Type your write in candidate Write In option failed to Blank box You will need to use the work properly. English alphabet. You will need to use the English The up and down arrows alphabet. You have 28 more characters did not work to get to left” You have 28 characters left alphabet to type in Write in candidate’s name. Voting Session #11 Type candidate name “Type your write in candidate Write In option failed to Blank box You will need to use the work properly. English alphabet. You will need to use the English The up and down arrows alphabet. You have 28 more characters did not work to get to left” You have 28 characters left alphabet to type in Write in candidate’s name. “Your selection is dash dash Using round button dash dash. resulted in ---- You have the maximum number of selections: Voting Session #12 Type candidate name “Type your write in candidate Write In option failed to Blank box You will need to use the work properly. English alphabet. You will need to use the English The up and down arrows alphabet. You have 28 more characters did not work to get to left” You have 28 characters left alphabet to type in Write Last update: 2019‐12‐24 Status: Final Version: 1.4 Page 14 of 60 FCMG: Accessibility Test Report Testing Logs Observation/ Screen Audio Physical task in candidate’s name. Using round button resulted in (Space) Pressed right arrow COUNTY Contest 6 of 15 Contest 6 of 15 JUDGE OF THE SUPERIOR COURT Contest Office No. 11 JUDGE OF THE SUPERIOR Vote for ONE Selections left: 1 COURT Office No. 11 List of candidates There are 4 options Vote for less than be greater than 1 be greater than / be greater than You have not made any selections on this contest Pressed down arrow and Type candidate name “Type your write in candidate then round button to Blank box You will need to use the scroll through candidates English alphabet. to the bottom choice of You will need to use the English Write in. alphabet. You have 28 more characters left” Audio read each You have 28 characters left candidates’ name and title but when going to Write in Candidate Audio did not read Write in selection. Screen went to Type candidate name screen. Pressed up and down “Type your write in candidate arrows and got Audio You will need to use the message. English alphabet. You have 28 more characters left” Last update: 2019‐12‐24 Status: Final Version: 1.4 Page 15 of 60 FCMG: Accessibility Test Report Testing Logs Observation/ Screen Audio Physical task Pressed round button while in Type candidate name box. Nothing occurred. Double clicking arrow and then round button allowed me to go from A to B. The logic for how letters are selected is not clear. You can go through the alphabet by multiple clicks of the up and down arrow keys. Delete can be accomplished with the left arrow key. It is difficult task to type a full name. Skipped contests by STATE CONTEST 13/15 CONTEST 13 of 15 using right arrow key to STATE MEASURE 50 State Contest 13/15 Vote YES or NO Selections left: 1 STATE MEASURE 50 YES on Measure 50 There are 2 options. NO on Measure 50 STATE MEASURE 50 Vote less than be greater than Yes less than / be greater than SUSPENSION OF LEGISLATORS, NO Less than be greater than LEGISLATIVE CONSTITIONAL NO /less AMENDMENT. Authorizes Legislature to suspend Member including without be greater salary and benefits. You have not made any Prohibits suspended Members from selections on this contest using powers of office or legislative resources. Provides suspension may end on specified date or by vote of Measure details Last update: 2019‐12‐24 Status: Final Version: 1.4 Page 16 of 60 FCMG: Accessibility Test Report Testing Logs Observation/ Screen Audio Physical task Member’s house. Fiscal Impact: No effect on state Less than be greater than be spending in most years. Minor state greater than suspension of savings in some years. LEGISLATORS, LEGISLATIVE CONSTITIONAL AMENDMENT. Less than /be greater than Authorizes Legislature to suspend Member including without salary and benefits. Prohibits suspended Members from using powers of office or legislative resources. Provides suspension may end on specified date or by vote of Member’s housel Fiscal Impact: (Note: No further Audio for ballot text) Press right arrow Voter Nominated Offices No Audio. (Note: No audio All voters, regardless of the party message to let voter preference they disclosed upon know what is happening registration… in the ballot) Press right arrow NATIONAL ELECTION Contest 14/15 Contest 14/15 NATIONAL ELECTION UNITED STATES SENATOR UNITED STATES SENATOR Vote for One Selections left:1 Vote for One Selections left:1 List of candidates below There are 34 options Vote for less than be greater than 1 be greater than / be greater than Last update: 2019‐12‐24 Status: Final Version: 1.4 Page 17 of 60 FCMG: Accessibility Test Report Testing Logs Observation/ Screen Audio Physical task You have not made any selections on this contest Contest 15/15 Contest 15/15 NATIONAL ELECTION NATIONAL ELECTION UNITED STATES SENATOR 1 UNITED STATES SENATOR 1 Let’s Review on the next screen you can review your selections before casting your ballot. Review your selections 10 contest with selections not made Ready to print? This is your last chance to go back and make any changes. Pressed down arrow Yes I am ready to print Pressed right arrow. Ballot printing Ballot printed and ejected Voter would not be Ready to cast “Ready to cast aware that there is a Verify your printed ballot before casting Verify your printed ballot before choice to either cast casting” ballot or have ballot read since this is not stated on (Note: This is not read on Audio Audio. Cast your ballot now – Read back my printed ballot Cast your ballot now Read back my printed ballot) Last update: 2019‐12‐24 Status: Final Version: 1.4 Page 18 of 60 FCMG: Accessibility Test Report Testing Logs Observation/ Screen Audio Physical task Up arrow gives audio message “Ready to cast? Verify printed ballot before casting. Pressed round button and Read back my printed ballot was selected. Voting Session #13 Ready to cast Ready to cast Verify your printed ballot before casting You have 2 options Cast your ballot or Cast your ballot now Read back your ballot. Read back my printed ballot To hear the first option, press the down arrow (Pressed the down arrow) To cast your ballot, Press the round button. To hear the next option, press the down Arrow. (Pressed the down arrow) To have your printed ballot read back to you press the round button To hear the instructions, press the down arrow. (Pressed the down arrow – Last update: 2019‐12‐24 Status: Final Version: 1.4 Page 19 of 60 FCMG: Accessibility Test Report Testing Logs Observation/ Screen Audio Physical task Nothing happened) (Pressed up arrow – went to cast your ballot now) (Pressed round button) You have chosen cast your ballot. Your ballot will be automatically placed into the ballot box and your vote will be finalized. To confirm your selection. press the right arrow. Pressed the right arrow. Casting your ballot. Your ballot is being placed in (Audio cut off) Thank you for voting. Don't forget your sticker on the way out. Pressed right arrow Are you sure you want to cast your No Audio ballot? Pressed arrow keys and no selection is made. Pressed round button Last update: 2019‐12‐24 Status: Final Version: 1.4 Page 20 of 60 FCMG: Accessibility Test Report Testing Logs Observation/ Screen Audio Physical task and it takes you back to Check your ballot Ballot read (all selections). At end of ballot pressed right arrow key and went to “Are you sure you want to cast your ballot? None of the arrow keys or round button will allow ballot to be cast. Pressed cast ballot on touch screen. Voting Session #14 Welcome to the June 01, 2016, No audio heard when PRESIDENTIAL PRIMARY ELECTION headphones were put on. Began voting by touching start 2016 CVSS Reference 3.2.8.b. Touch the circle to start (Multiple languages scrolling through under English text) For audio, please put on the headphones. I’m confused about what I’m confused about what to do I’m confused about what to do to do Making your selection Making your selection To make your selection, touch your To make your selection, touch choice. A check mark will appear in your choice. A check mark will your selection appear in your selection Changing your selection Changing your selection To make a change, touch the selection To make a change, touch the you no longer want. Then touch the selection you no longer want. Last update: 2019‐12‐24 Status: Final Version: 1.4 Page 21 of 60 FCMG: Accessibility Test Report Testing Logs Observation/ Screen Audio Physical task choice you do want. Then touch the choice you do want. You do not have to vote in You do not have to vote in every every contest contest If you don’t want to vote in a contest, press the “Skip” button to move If you don’t want to vote in a forward to the next contest. contest, press the “Skip” button to move forward to the next contest. Turn screen on/off Turn screen on/off To Turn the screen off and hide your selections press the round Screen on button Screen off To keep the screen on and show your selections press the right arrow. I want to quit the voting Are you sure you want to quit your Are you sure you want to quit session voting session? your voting session? After you quit, talk to an election worker. No – I want to go back Your vote will not be counted Yes – Quit my voting session until you cast your ballot. After you quit, talk to an election worker. Your vote will not be counted until you case your ballot If this is not To hear the first option, press your session, contact an election the down arrow. worker for help. To go back to the previous page, press the round button. To hear the next option, press Last update: 2019‐12‐24 Status: Final Version: 1.4 Page 22 of 60 FCMG: Accessibility Test Report Testing Logs Observation/ Screen Audio Physical task the down arrow. (“No – I want to go back” - is highlighted) To quit the voting session, press the round button After pressing the down button No – I want to go back Yes – Quit my voting session After you quit, talk to an election worker. Your vote will not be counted until you cast your ballot If this is not your session, contact an election worker for help. I want help from an Ask an election worker what to do next Ask an election worker what to election worker do next Bring your ballot with you if you have one. Bring me (unintelligible) your ballot with you if you have one. I want help from an Ask an election worker what to do next Ask an election worker what to election worker do next Bring your ballot with you if you have one. Bring me (unintelligible) your ballot with you if you have one. There seems to be a problem Last update: 2019‐12‐24 Status: Final Version: 1.4 Page 23 of 60 FCMG: Accessibility Test Report Testing Logs Observation/ Screen Audio Physical task Ballot ejected but no ballot came out. Voting Session #15 Shows: Began voting session County Seal System Ready Tapped the screen Tap anywhere to begin Entered credentials Selected Shows: Reopen the Polls System Status 2016 General Election Printing Poll Report Start Voting System Status Began voting session – Shows: Audio is automatically set to very low level. Put on headset. Welcome to the November 08, 2016, GENERAL ELECTION 2016 Touched circle. Touch the circle to start Touched volume + button and it went to the next screen that reads Choose language. Audio states: CHOOSE YOUR LANGUAGE ON THE TOUCH SCREEN OR USE THE CONTROLLER TO Last update: 2019‐12‐24 Status: Final Version: 1.4 Page 24 of 60 FCMG: Accessibility Test Report Testing Logs Observation/ Screen Audio Physical task CONTINUE IN ENGLISH To hear the next language, push the down arrow. Pushed the down arrow button to scroll through the languages and returned to English and selected to continue Voting Session Choose language Audio states: English You have chosen English. To confirm English press, the next button on the touch screen or right arrow on the controller. To hear the next language, press the down arrow. Selected right arrow Inserted blank ballot with Insert your ballot Audio states: notched corner on top Insert your ballot into the tray to your Insert your ballot right into the paper tray right feed. Insert your ballot into the paper tray to your right. On your ballot one corner has been cut off. Hold your ballot with the cut corner at the top right and insert it into the paper tray until it gets pulled inside. Last update: 2019‐12‐24 Status: Final Version: 1.4 Page 25 of 60 FCMG: Accessibility Test Report Testing Logs Observation/ Screen Audio Physical task After ballot is pulled Your ballot is empty. Audio states: inside then screen Ask an election worker to help Loading your ballot displays message. manually enter ballot information. Please wait a moment Audio does not tell voter what to do next. There is silence. CVSS Reference: 3.2.5.a. Scanned Vote Mode Enter Credentials None Code Typed in credentials Proceed to enter Precinct Entered Precinct number Precinct 7770001A is a valid precinct. None To enter a different number, press Back. If this precinct is correct, press Next Pressed Next. Selected No. Is this ballot provisional? None Then, Selected Next. Yes or No How to use the controller Audio states: Picture Let’s go through how to use the controller. Next Hold the controller so the cord Last update: 2019‐12‐24 Status: Final Version: 1.4 Page 26 of 60 FCMG: Accessibility Test Report Testing Logs Observation/ Screen Audio Physical task faces away from you. To skip these instructions at any time, press the right arrow There are 4 arrows in the middle with the round button on the right Use these buttons to navigate through the ballot. At the beginning of each contest you can press the left arrow to go back to the previous page At the candidate list you can press the left arrow to repeat the previous instructions The help button is located on the top left. The audio volume buttons are on the top right. The play back rate is on the bottom right. The pause/play button is on the bottom left. To continue, press the right arrow Turn screen on/off Audio states: Screen on To turn the screen off and hide Last update: 2019‐12‐24 Status: Final Version: 1.4 Page 27 of 60 FCMG: Accessibility Test Report Testing Logs Observation/ Screen Audio Physical task Screen off your selections press the round button Back Next To keep the screen on and show your selections Press the right arrow Pressed the right arrow Let’s get started Let’s get started You have 2 options Start voting or use a poll pass To hear the first option press the down arrow Pressed the down arrow To start voting and navigate through the ballot Press the round button To hear the next option Press the down arrow Pressed the down arrow If you have a poll pass press the round button To hear the instructions press the down arrow Pressed the down arrow Let’s get started You have 2 options Start voting or use a poll pass To hear the first option press the down arrow Pressed the down arrow To start voting and navigate through the ballot press the Last update: 2019‐12‐24 Status: Final Version: 1.4 Page 28 of 60 FCMG: Accessibility Test Report Testing Logs Observation/ Screen Audio Physical task round button Pressed the round button You have chosen to start voting and navigate through the ballot To confirm your selection Press the right arrow Pressed the right arrow Make your selections Make your selections There are 29 contests on today’s ballot There are 29 contests on today’s ballot When you reach the end, you will be able to review all your selections. When you reach the end you will be able to review all your selections. To go to the first contest Press the right arrow Proceeded by pressing CITY/LOCAL Audio states: right arrow to go to next STATE SENATOR. 27th District Ding screen Vote for ONE Selections left:1 Contest 1 of 29 City/Local Note: Names of candidates State senator 27th district displayed There are 2 options Vote for one Back Skip You have not made any selections on this contest To hear the first option Press the down arrow. Last update: 2019‐12‐24 Status: Final Version: 1.4 Page 29 of 60 FCMG: Accessibility Test Report Testing Logs Observation/ Screen Audio Physical task To go to the next contest press the right arrow. To go back to the previous page, press the left arrow Pressed the down arrow To select Steve Fazio Party Preference Republican Independent small business man Press the round button To hear the next option Press the down arrow To go to the next contest Press the right arrow Pressed the down arrow To select Henry Stern Party Preference Democrat Educator/Environmental Attorney Press the round button To hear the next option Press the right arrow To go to the next contest Press the right arrow Last update: 2019‐12‐24 Status: Final Version: 1.4 Page 30 of 60 FCMG: Accessibility Test Report Testing Logs Observation/ Screen Audio Physical task Pressed the round button Screen shows Henry Stern with You have selected checkmark and in black highlight. Henry Stern You have the maximum number of selections To deselect Henry Stern press the round button. To hear the next option Press the down arrow To go to the next contest Press the right arrow. Pressed right arrow Contest 2/29 Contest 2 of 29 CITY/LOCAL CITY/LOCAL MEMBER OF THE STATE MEMBER OF THE STATE ASSEMBLY, 44th District. ASSEMBLY, 44th District. Vote for ONE There are 2 options Selections Left:1 Vote for ONE You have not made any Selections on this contest To hear the first option Press the down arrow. To go to the next contest Press the right arrow. To go back to the previous page Last update: 2019‐12‐24 Status: Final Version: 1.4 Page 31 of 60 FCMG: Accessibility Test Report Testing Logs Observation/ Screen Audio Physical task press the left arrow. Pressed the down arrow To select Kerry J. Nelson Party Preference Republican Small Business Owner Press the round button. To hear the next option Press the down arrow To go to the next contest Press the right arrow. Pressed the round button You have selected Kerry J. Nelson You have the maximum number of selections To deselect Kerry J. Nelson press the round button. To hear the next option Press the down arrow To go to the next contest Press the right arrow. Pressed right arrow County Contest 6/29 Contest Then proceeded by using JUDGE OF THE SUPERIOR COURT 6 of 29 right arrow to go to Office No. 42 County Contest 6 JUDGE OF THE SUPERIOR Last update: 2019‐12‐24 Status: Final Version: 1.4 Page 32 of 60 FCMG: Accessibility Test Report Testing Logs Observation/ Screen Audio Physical task COURT Office No. 42 There are 2 options Vote for 1 You have not made any selections on this contest To hear the first option Press the down arrow. To go to the next contest Press the right arrow. To go back to the previous page press the left arrow. Pressed the down arrow To select Alicia Molina Attorney at law Press the round button To hear the next option Press the down arrow To go to the next contest Press the right arrow. Pressed the round button You have selected Alicia Molina You have the maximum number of selections To deselect Last update: 2019‐12‐24 Status: Final Version: 1.4 Page 33 of 60 FCMG: Accessibility Test Report Testing Logs Observation/ Screen Audio Physical task Alicia Molina press the round button. To hear the next option Press the down arrow To go to the next contest Press the right arrow. Pressed the right arrow County Contest 7/29 Contest JUDGE OF THE SUPERIOR COURT 7 of 29 Office No. 84 County Vote for ONE JUDGE OF THE SUPERIOR Selections left:1 COURT Office No. 84 Names of candidates There are 2 options Vote for 1 You have not made any selections on this contest To hear the first option Press the down arrow. To go to the next contest Press the right arrow. To go back to the previous page press the left arrow. Pressed the down arrow To select To select Susan Jung Townsend Susan Jung Townsend Press the round button Last update: 2019‐12‐24 Status: Final Version: 1.4 Page 34 of 60 FCMG: Accessibility Test Report Testing Logs Observation/ Screen Audio Physical task To hear the next option Press the down arrow To go to the next contest Press the right arrow To select Javier Perez Supervising Criminal Prosecutor Press the round button To hear the next option Press the down arrow To go to the next contest Press the right arrow To write in your own candidate name Press the round button To hear the next option Press the down arrow To go to the next contest Press the right arrow Type your write in candidate You have 28 characters left Your write in is GISELA Last update: 2019‐12‐24 Status: Final Version: 1.4 Page 35 of 60 FCMG: Accessibility Test Report Testing Logs Observation/ Screen Audio Physical task To erase the last letter, press the left arrow To confirm your write in press the right arrow Pressed right arrow Your write in GISELA Is currently selected You have the maximum number of selections To deselect GISELA press the round button. To hear the next option Press the down arrow To go to the next contest Press the right arrow. Pressed right arrow Then proceeded by using right arrow to go to last race. Let’s Review Let’s Review On the next screen you can review On the next screen you can your selections before casting your review your selections before ballt. casting your ballot You have completed all of the Last update: 2019‐12‐24 Status: Final Version: 1.4 Page 36 of 60 FCMG: Accessibility Test Report Testing Logs Observation/ Screen Audio Physical task contests on this ballot To review your selections, press the right arrow To return to the ballot Press the left arrow Review your selections. Review your selections. 12 contest with selections not made. To review your selections Press the down arrow 25 contests with selections not made. Proceeded by pressing State Senator, the down arrow. 27th district Went through all races to Your selection is last contest. Henry Stern You have the maximum number of selections. To change this selection, press the round button. To review the next contest Press the down arrow. To finish the review Press the right arrow Pressed the down arrow End of options. To hear the review instructions Last update: 2019‐12‐24 Status: Final Version: 1.4 Page 37 of 60 FCMG: Accessibility Test Report Testing Logs Observation/ Screen Audio Physical task press the down arrow. Pressed the right arrow Ready to print? Ready to print t This is your last chance to go back and This is your last chance to make any changes. make any changes. Yes – I am ready to print. To print and verify your ballot before casting Press the round button Note: This was fixed from primary version of ballot that had no instructions on how to review your selections. Proceeded by You have chosen to print the ballot. Pressing round button To confirm your selection Press the right arrow Pressed right arrow Printing ballot Ding ding Printing ballot Ballot printed Ready to cast You have 2 options Cast your ballot or read back your ballot To hear the first option Press the down arrow Last update: 2019‐12‐24 Status: Final Version: 1.4 Page 38 of 60 FCMG: Accessibility Test Report Testing Logs Observation/ Screen Audio Physical task Pressed the down arrow To cast your ballot Press the round button To hear the next option Press the down arrow Pressed the down arrow To have your printed ballot read back to you Press the round button. To hear the instructions Press the down arrow. Pressed the round Ready to cast? You have chosen read back my button. ballot Very your printed ballot before casting. The machine will scan your Cast your ballot now ballot and read back your selections. Read back my printed ballot To confirm your selection press the right arrow. Pressed the right arrow Ding ding Reading back your ballot Did nothing. The system could not read back your ballot Waited for machine to scan ballot. Ballot reloaded. Ballot reloaded Loading Ballot Loading your ballot please wait a moment Please wait a moment For audio, please put on the Last update: 2019‐12‐24 Status: Final Version: 1.4 Page 39 of 60 FCMG: Accessibility Test Report Testing Logs Observation/ Screen Audio Physical task headphones. Ballot could not be read Ballot could not be read The system could not read the ballot. Then went to: Then continued to screen: Welcome to the November 8, 2016 General Election Welcome to the November 8, 2016 General Election Tried reloading ballot. Loading Ballot Please wait a moment For audio, please put on the headphones. After 3 minutes, received Ballot could not be read The system could not read the message. ballot. Then went to: Please reinsert the ballot. Welcome to the November 8, 2016 General Election Welcome to the November 8, 2016 General Election To begin touch the circle on the screen or press any button on the controller. Loaded ballot Loading Ballot Loading your ballot Please wait a moment Loading your ballot Please wait a minute. After 3 minutes, received Ballot could not be read The system could not read the message. ballot. Then went to: Please reinsert the ballot. Welcome to the November 8, 2016 General Election Last update: 2019‐12‐24 Status: Final Version: 1.4 Page 40 of 60 FCMG: Accessibility Test Report Testing Logs Observation/ Screen Audio Physical task Welcome to the November 8, 2016 General Election To begin touch the circle on the screen or press any button on the controller. Went to another BMD to load ballot. Proceeded to open polls. BMD screen went black and blank ballot for Open Poll Report was stuck midway in tray. Went to another BMD to load ballot. Loaded ballot Loading Ballot Loading your ballot Please wait a moment Loading your ballot Please wait a minute. After 3 minutes, received Ballot could not be read The system could not read the message. ballot. Then went to: Please reinsert the ballot. Welcome to the November 8, 2016 General Election Welcome to the November 8, 2016 General Election To begin touch the circle on the screen or press any button on the controller. Spoke to Ed Smith, Smartmatic about Last update: 2019‐12‐24 Status: Final Version: 1.4 Page 41 of 60 FCMG: Accessibility Test Report Testing Logs Observation/ Screen Audio Physical task experience. Ballot had been printed on 8.5 x 11 paper. General Ballot is set for 8.5 x 14 paper and therefore the ballot is truncated and cannot be read. End of Session. Voting Session #16 Shows: Began voting session County Seal System Ready Tapped the screen Tap anywhere to begin Entered credentials Selected Shows: Reopen the Polls System Status 2016 General Election Printing Poll Report Start Voting System Status Began voting session – Shows: Choose language. Put on headset. Welcome to the November 08, 2016, GENERAL ELECTION 2016 Audio states: CHOOSE YOUR LANGUAGE Last update: 2019‐12‐24 Status: Final Version: 1.4 Page 42 of 60 FCMG: Accessibility Test Report Testing Logs Observation/ Screen Audio Physical task Touched circle. Touch the circle to start ON THE TOUCH SCREEN OR USE THE CONTROLLER TO CONTINUE IN ENGLISH To hear the next language, push the down arrow. Pushed the down arrow button to scroll through the languages and returned to English and selected to continue Voting Session #2 Choose language Audio states: English You have chosen English. To confirm English press the next button on the touch screen or right arrow on the controller. To hear the next language, press the down arrow. Selected right arrow Inserted blank ballot with Insert your ballot Audio states: notched corner on top Insert your ballot into the tray to your Insert your ballot right into the paper tray right feed. Insert your ballot into the paper tray to your right. On your ballot one corner has been cut off. Hold your ballot with the cut corner at the top right and insert it into the paper tray until it gets pulled inside. Last update: 2019‐12‐24 Status: Final Version: 1.4 Page 43 of 60 FCMG: Accessibility Test Report Testing Logs Observation/ Screen Audio Physical task Ballot is pulled inside Your ballot is empty. Audio states: then screen displays Ask an election worker to help Loading your ballot message. manually enter ballot information. Please wait a moment Audio does not tell voter what to do next. There is silence. CVSS Reference: 3.2.5.a. Scanned Vote Mode Enter Credentials None Code Typed in credentials Proceed to enter Precinct Entered Precinct number Precinct 7770001A is a valid precinct. None To enter a different number, press Back. If this precinct is correct, press Next Pressed Next. Selected No. Is this ballot provisional? None Then, Selected Next. Yes or No How to use the controller Audio states: Picture Let’s go through how to use the controller. Next Last update: 2019‐12‐24 Status: Final Version: 1.4 Page 44 of 60 FCMG: Accessibility Test Report Testing Logs Observation/ Screen Audio Physical task Hold the controller so the cord faces away from you. To skip these instructions at any time, press the right arrow There are 4 arrows in the middle with the round button on the right Use these buttons to navigate through the ballot. At the beginning of each contest you can press the left arrow to go back to the previous page At the candidate list you can press the left arrow to repeat the previous instructions The help button is located on the top left. The audio volume buttons are on the top right. The play back rate is on the bottom right. The pause/play button is on the bottom left. To continue, press the right arrow Turn screen on/off Audio states: Last update: 2019‐12‐24 Status: Final Version: 1.4 Page 45 of 60 FCMG: Accessibility Test Report Testing Logs Observation/ Screen Audio Physical task Screen on To turn the screen off and hide your selections press the round Screen off button Back Next To keep the screen on and show your selections Press the right arrow Pressed the right arrow Let’s get started Let’s get started You have 2 options Start voting or use a poll pass To hear the first option press the down arrow Pressed the down arrow To start voting and navigate through the ballot Press the round button To hear the next option Press the down arrow Pressed the down arrow If you have a poll pass press the round button To hear the instructions press the down arrow Pressed the down arrow Let’s get started You have 2 options Start voting or use a poll pass To hear the first option press the down arrow Last update: 2019‐12‐24 Status: Final Version: 1.4 Page 46 of 60 FCMG: Accessibility Test Report Testing Logs Observation/ Screen Audio Physical task Pressed the down arrow To start voting and navigate through the ballot press the round button Pressed the round button You have chosen to start voting and navigate through the ballot To confirm your selection Press the right arrow Pressed the right arrow Make your selections Make your selections There are 29 contests on today’s ballot There are 29 contests on today’s ballot When you reach the end, you will be able to review all your selections. When you reach the end you will be able to review all your selections. To go to the first contest Press the right arrow Proceeded by pressing CITY/LOCAL Audio states: right arrow to go to next STATE SENATOR. 27th District Ding screen Vote for ONE Selections left:1 Contest 1 of 29 City/Local SPANISH Note: Names of candidates State senator 27th district displayed SPANISH There are 2 options Back Skip Vote for one SPANISH You have not made any selections on this contest To hear the first option Last update: 2019‐12‐24 Status: Final Version: 1.4 Page 47 of 60 FCMG: Accessibility Test Report Testing Logs Observation/ Screen Audio Physical task Press the down arrow. To go to the next contest press the right arrow. To go back to the previous page, press the left arrow Pressed the down arrow To select Steve Fazio Party Preference Republican SPANISH Independent small business man SPANISH Press the round button To hear the next option Press the down arrow To go to the next contest Press the right arrow Pressed the down arrow To select Henry Stern SPANISH Party Preference Democrat SPANISH Educator/Environmental Attorney SPANISH Press the round button To hear the next option Press the right arrow Last update: 2019‐12‐24 Status: Final Version: 1.4 Page 48 of 60 FCMG: Accessibility Test Report Testing Logs Observation/ Screen Audio Physical task To go to the next contest Press the right arrow Pressed the round button Screen shows Henry Stern with You have selected checkmark and in black highlight. Henry Stern SPANISH You have the maximum number of selections To deselect Henry Stern SPANISH press the round button. To hear the next option Press the down arrow To go to the next contest Press the right arrow. Pressed right arrow Contest 2/29 Contest 2 of 29 CITY/LOCAL CITY/LOCAL SPANISH MEMBER OF THE STATE MEMBER OF THE STATE ASSEMBLY, 44th District. ASSEMBLY, 44th District SPANISH Vote for ONE Selections Left:1 There are 2 options Vote for ONE SPANISH You have not made any Selections on this contest To hear the first option Press the down arrow. Last update: 2019‐12‐24 Status: Final Version: 1.4 Page 49 of 60 FCMG: Accessibility Test Report Testing Logs Observation/ Screen Audio Physical task To go to the next contest Press the right arrow. To go back to the previous page press the left arrow. Pressed the down arrow To select Kerry J. Nelson SPANISH Party Preference Republican SPANISH Small Business Owner SPANISH Press the round button. To hear the next option Press the down arrow To go to the next contest Press the right arrow. Pressed the round button You have selected Kerry J. Nelson SPANISH You have the maximum number of selections To deselect Kerry J. Nelson SPANISH press the round button. To hear the next option Press the down arrow Last update: 2019‐12‐24 Status: Final Version: 1.4 Page 50 of 60 FCMG: Accessibility Test Report Testing Logs Observation/ Screen Audio Physical task To go to the next contest Press the right arrow. Pressed right arrow County Contest 6/29 Contest Then proceeded by using JUDGE OF THE SUPERIOR COURT 6 of 29 right arrow to go to Office No. 42 County Contest 6 JUDGE OF THE SUPERIOR COURT Office No. 42 There are 2 options Vote for 1 You have not made any selections on this contest To hear the first option Press the down arrow. To go to the next contest Press the right arrow. To go back to the previous page press the left arrow. Pressed the down arrow To select Alicia Molina Attorney at law Press the round button To hear the next option Press the down arrow To go to the next contest Last update: 2019‐12‐24 Status: Final Version: 1.4 Page 51 of 60 FCMG: Accessibility Test Report Testing Logs Observation/ Screen Audio Physical task Press the right arrow. Pressed the round button You have selected Alicia Molina You have the maximum number of selections To deselect Alicia Molina press the round button. To hear the next option Press the down arrow To go to the next contest Press the right arrow. Pressed the right arrow County Contest 7/29 Contest JUDGE OF THE SUPERIOR COURT 7 of 29 Office No. 84 County Vote for ONE JUDGE OF THE SUPERIOR Selections left:1 COURT Office No. 84 Names of candidates There are 2 options Vote for 1 You have not made any selections on this contest To hear the first option Press the down arrow. To go to the next contest Press the right arrow. Last update: 2019‐12‐24 Status: Final Version: 1.4 Page 52 of 60 FCMG: Accessibility Test Report Testing Logs Observation/ Screen Audio Physical task To go back to the previous page press the left arrow. Pressed the down arrow To select To select Susan Jung Townsend Press the round button To hear the next option Press the down arrow To go to the next contest Press the right arrow To select Javier Perez Supervising Criminal Prosecutor Press the round button To hear the next option Press the down arrow To go to the next contest Press the right arrow To write in your own candidate name Press the round button To hear the next option Press the down arrow To go to the next contest Last update: 2019‐12‐24 Status: Final Version: 1.4 Page 53 of 60 FCMG: Accessibility Test Report Testing Logs Observation/ Screen Audio Physical task Press the right arrow Type your write in candidate You have 28 characters left Your write in is GISELA To erase the last letter, press the left arrow To confirm your write in press the right arrow Pressed right arrow Your write in GISELA Is currently selected You have the maximum number of selections To deselect GISELA press the round button. To hear the next option Press the down arrow To go to the next contest Press the right arrow. Pressed right arrow Then proceeded by using right arrow to go to last Last update: 2019‐12‐24 Status: Final Version: 1.4 Page 54 of 60 FCMG: Accessibility Test Report Testing Logs Observation/ Screen Audio Physical task race. Let’s Review Let’s Review On the next screen you can review On the next screen you can your selections before casting your review your selections before ballot. casting your ballot You have completed all of the contests on this ballot To review your selections, press the right arrow To return to the ballot Press the left arrow Review your selections. Review your selections. 12 contest with selections not made. To review your selections Press the down arrow 25 contests with selections not made. Proceeded by pressing State Senator, the down arrow. 27th district Went through all races to Your selection is last contest. Henry Stern You have the maximum number of selections. To change this selection, press Last update: 2019‐12‐24 Status: Final Version: 1.4 Page 55 of 60 FCMG: Accessibility Test Report Testing Logs Observation/ Screen Audio Physical task the round button. To review the next contest Press the down arrow. To finish the review Press the right arrow Pressed the down arrow End of options. To hear the review instructions press the down arrow. Pressed the right arrow Ready to print? Ready to print t This is your last chance to go back and This is your last chance to make any changes. make any changes. Yes – I am ready to print. To print and verify your ballot before casting Press the round button Note: This was fixed from primary version of ballot that had no instructions on how to review your selections. Proceeded by You have chosen to print the ballot. Pressing round button To confirm your selection Press the right arrow Pressed right arrow Printing ballot Ding ding Last update: 2019‐12‐24 Status: Final Version: 1.4 Page 56 of 60 FCMG: Accessibility Test Report Testing Logs Observation/ Screen Audio Physical task Printing ballot Ballot printed Ready to cast You have 2 options Cast your ballot or read back your ballot To hear the first option Press the down arrow Pressed the down arrow To cast your ballot Press the round button To hear the next option Press the down arrow Pressed the down arrow To have your printed ballot read back to you Press the round button. To hear the instructions Press the down arrow. Pressed the round Ready to cast? You have chosen read back my button. ballot Very your printed ballot before casting. The machine will scan your Cast your ballot now ballot and read back your selections. Read back my printed ballt To confirm your selection press the right arrow. Pressed the right arrow Ding ding Reading back your ballot Last update: 2019‐12‐24 Status: Final Version: 1.4 Page 57 of 60 FCMG: Accessibility Test Report Testing Logs Observation/ Screen Audio Physical task Did nothing. The system could not read back your ballot Waited for machine to scan ballot. Ballot reloaded. Ballot reloaded Loading Ballot Loading your ballot please wait a moment Please wait a moment For audio, please put on the headphones. Ballot could not be read Ballot could not be read The system could not read the ballot. Then went to: Then continued to screen: Welcome to the November 8, 2016 General Election Welcome to the November 8, 2016 General Election Tried reloading ballot. Loading Ballot Please wait a moment For audio, please put on the headphones. After 3 minutes, received Ballot could not be read The system could not read the message. ballot. Then went to: Please reinsert the ballot. Welcome to the November 8, 2016 General Election Welcome to the November 8, 2016 General Election To begin touch the circle on the screen or press any button on the controller. Last update: 2019‐12‐24 Status: Final Version: 1.4 Page 58 of 60 FCMG: Accessibility Test Report Testing Logs Observation/ Screen Audio Physical task Loaded ballot Loading Ballot Loading your ballot Please wait a moment Loading your ballot Please wait a minute. After 3 minutes, received Ballot could not be read The system could not read the message. ballot. Then went to: CVSS Reference Please reinsert the ballot. Welcome to the November 8, 2016 3.2.8.b. General Election Welcome to the November 8, 2016 General Election To begin touch the circle on the screen or press any button on the controller. Went to another BMD to load ballot. Proceeded to open polls. BMD screen went black and blank ballot for Open Poll Report was stuck midway in tray. Went to another BMD to load ballot. Loaded ballot Loading Ballot Loading your ballot Please wait a moment Loading your ballot Last update: 2019‐12‐24 Status: Final Version: 1.4 Page 59 of 60 FCMG: Accessibility Test Report Testing Logs Observation/ Screen Audio Physical task Please wait a minute. After 3 minutes, received Ballot could not be read The system could not read the message. ballot. Then went to: CVSS Reference Please reinsert the ballot. Welcome to the November 8, 2016 3.2.8.b. General Election Welcome to the November 8, 2016 General Election To begin touch the circle on the screen or press any button on the controller. Spoke to Ed Smith about experience. Ballot had been printed on 8.5 x 11 paper. General Ballot is set for 8.5 x 14 paper and therefore the ballot is truncated and cannot be read. End of Session. Last update: 2019‐12‐24 Status: Final Version: 1.4 Page 60 of 60