A semiautomatic approach for testing the accessibility of web pages
Pelzetter, Jens
Creating web pages has become a challenging task in the last years. Today web pages and web applications have to work a large variety of devices. On top of these requirements web developers have also to care about accessibility. The primary guidelines for accessible web pages are the Web Content Accessibility Guidelines published by the W3C. These guidelines define several requirements for accessible web pages. The W3C also publishes recommendations how these guidelines can be implemented. Currently a new standard is in development which specifies a format for writing rules for testing the accessibility of web pages and to harmonize the interpretation of the guidelines. Nevertheless, evaluating the accessibility of a web page is a time consuming task. The process can also be error prone, depending on the experience of the user. This talk will present the prototype of semiautomatic tool for evaluating the accessibility of web pages. The tool uses a combination of automatic and guided manual testing to make accessibility evaluations less time consuming and error prone. For implementing the tool several well known projects like Selenium are used. The first part of the talk will give a brief overview about the standards for accessible web pages and the procedures for testing the accessibility of web pages. One of the main challenges in the development of the tool was the modeling of the rules for accessible web pages in a machine readable way. In the main part of the talk the model used for the rules as well as the implementation of the test procedures will be discussed. The last part of the talk will be a live demonstration of the of the tool presented in this talk.
 So, hello everyone. We have now the second talk in the science meet, Froskon meet science track and I'm introduced to you Jens Pelseter from the University of Bremen, right? And he's right now doing his PhD and the topic he is talking today is part of his, of his PhD and he talks about the automatic approach for testing the accessibility of web pages. Truly an important issue, nevertheless maybe one issue not on the top list of the developers. So it's your stage. Yeah, thank you. Okay. Yeah. But it describes the thing I want to talk about. It's best. First some words about accessibility about web pages. Most people think about blind people if they hear this topic. That's not the truth, the complete truth. Blind people are only one part of accessibility. An important part nevertheless, but that there are many more impairments which can affect the way you can use web pages. One example about 8% of the male population can't see red and green as colors. So that's not too small amount of people. And what are accessible web pages? There are a few standards in this. The most important one are the web content accessibility guidelines which are available in the version 2.1 since last year. They are organized into four principles. Percivable, operable, understandable and robust. Which then are split into 13 guidelines and 78 success criteria which describe where the various requirements for accessible web pages. There are also some supplemental documents understanding the way content accessibility guidelines which describes the intent of the success criteria in detail and the techniques for web content accessibility guidelines describes how you can implement them. The next thing is not that easy. But I will come to that later. There are a few other standards. For example the authoring tool accessibility guidelines or the accessible rich internet application standards. If you are interested in that I will give a talk in C119 this afternoon about these standards where I will talk in detail about them. So if you are interested it's at 2 o'clock this afternoon. The testing and web page accessibility is quite complex task. It requires manual work at the moment. Even there are some browser plugins which you can use to test some parts or some aspects of the accessibility but not everything. It still requires very manual work and it's very time consuming and the guidelines from the web content accessibility guidelines are also open for interpretation. The W3C has recognized this and is currently developing a new standard. Accessibility conformance testing rules which defines a format for describing accessibility conformance testing rules. And first rules using the standards are available at a GitHub repository. The second is that it's still a textual description so nothing which you can use for automatic tests. The intent is to provide a tool implementers a guideline for these things but at the moment there are not very much tools which implement that. And the idea which I had is you should automate as much as possible if you test these and for the steps which require manual work you should guide the users to these steps so that they know exactly what to check for. And I developed as part of my PhD tool which is structured like this. First there is a service which has an ontology with the rules and executes the test using Selenium. I don't know if Selenium is, if you know Selenium that's a tool which you can use for testing web user interfaces. It's often used to check if a user interface works correctly. And the service uses Selenium to do that, provides a user interface or the web user interface. And the real testing is done by multiple instances of a bot which then executes the Selenium test and tests the web side. The web user interface looks at a moment like this, it's all implemented in Java because the reason for that is that APIs for using ontologies are only available in Java. So the easy way was to do the rest also in Java. If you want to evaluate a web page you simply put the URL here, click start and then the tool starts to run, analyzes the document and executes all tests it knows. It first determines which rules are applicable for each element and then executes for each element the applicable test. And at the end you get some screen like this, this way you see the task which requires manual work and if you then click on the button you see a screenshot with the element and the question and simply click past or failed. And at the end, when all tests are finished, you get an overview about the rules. For this page many rules are not applicable. And here you see some rules which have passed or failed and that's it. The short description of these Act Rules looks like this. Yeah, so here that's the description when this rule is applicable. Don't need to read this now. And in my tool the translation looks like this if you put that in the pseudo code. The rule has three conditions which the element must match so that the rule is applicable. And in this case it should be a button, should be exposed to the accessibility tree. Every browser builds an additional tree of the elements of a web page which is available for assistive technologies, for example screen readers. And not all elements are exposed to this tree. And this element should not be an input element with the type image. Which yeah. And now if you are lucky. Yeah, it still runs. Before the talk I started evaluation of the FrostCon web page which is still running. That's also one problem which I'm currently working on. It takes very, at the moment the testing takes very long because the problem is a web page has, a normal web page has many elements. FrostCon web page has not so much but for example the start page of our university has about 2000 HTML elements. And at the moment it takes the tool needs very long to test them all. But I think I will, I can cut this time so that it takes about half an hour for such a large page then to test. It's still a long time but if you want to evaluate such a large page manually you will need longer. Because you have to look for each element and that for each tool. So the tool will save you time. And yeah. I think the tool also has some problem. Okay, yeah. That's so far. Yeah, that's the current status. I'm currently working on bug fix and optimizations. And I'm looking for possible participants which want to evaluate this tool if it's finished. So if you're interested please write me in mail and I will get back to you when the tool is ready. Yeah. Then some other talks. I will give two other talks here on the FrostCon one this afternoon in C119 which where I will talk about the standards for accessible web pages. And tomorrow I will give a talk where I will explain why accessible web pages are also useful for everybody. Not only people with impairments. And some other advertisements. I'm also a member of the CMS garden which has also a boot here. And we have our own unconference in November. So if you want to meet us and learn about web content management systems you can come to Essen. Attendance is free as here. Yeah. Okay, questions. Yeah. You started the scan from the web page. So you started the scan from a web page. Is there also the possibility to start it from command line? No. So that would enable. Not yet. Okay. Maybe I will work on that. What you can do is to use the browser extension. So you can start it directly from the browser. But that extension is still in development. The reason I'm asking is if you could integrate this into your CI pipeline that would be much more pleasant because you don't have to wait half an hour or however long it takes on your local computer. But you can just push to the server, have it build it. You can leave this web page. It's not necessary to leave it open. Yeah, but it still has to be running on my local computer. No, no, no, no. It's a web service. It will run on a server. On a server. But if I have that one instance of the page that I want to test and locally I want to continue development, it's much easier if I have multiple versions. Yeah. No, no, that's a second step to integration in CI pipelines. Okay. Thank you. Yeah. My question is in which different output formats can you get the report? At the moment, the question was in which out. Okay. At the moment, you can only get an output for humans. When the tool is finished, there is a standard for accessibility evaluation reports in a JSON format. Which you can get the report then as export. So as JSON, maybe other formats also, but the standard will be implemented. Jason would be quite fine. Thank you. Yeah. Thanks. What experience did you already get with this tool? So have you run it yourself on various websites? Or had many other projects that tried it? So far the results were that many webpages have some accessibility problems. Sometimes in parts which you don't expect. For example, one of the rules is that every element must have an accessible name which is exposed to this assistive technologies and not much webpages have that for all elements where it's necessary. It's not only text for images, but also real labels for input elements. I've seen many examples where you have a visual label for an input element, but it's not connected with the input in a way which can be recognized by assistive technology. Yeah. My question would be whether you have to specify each page on your website? At the moment, yes. That's also one of the next steps to automate the scan for a whole site. It's now the second step. First, get it running for a single document and then look into how you can do it for multiple pages. Okay, I have one more. Does it also work with, let's say, JavaScript intensive pages that have only small static parts and do a lot of rendering based on information that comes back after just some JavaScript calls? It should. That's also one thing which has to be tested, how it works. Maybe you will have to define some interactions which the tool should do in the background. I don't know if you know Selenium. In the background, the bot starts a Firefox box, so JavaScript is supported in this way, but maybe you will have to define some interactions with the page to get the complete rendering. Only JavaScript-based web user interfaces are really multiple pages, even if it's one page. So that's also one of the next steps if the tool is running fine for normal static web pages. Any more questions? How is it with reliability of the tests? Do you specify your rules? Will it get through the cases? Thankfully, the people which do define these act rules, which I'm currently using as base, also define test cases for tools which implement them. So that should be easy to check. That's also one of the parts which I'm currently working on. As you maybe know, you find many problems when you start testing and when you are developing such novel tools. So sometimes you are not as fast with the development as you want. I hope to show that I could show more today, but unfortunately there are some nasty bugs which I have to address in parts where I didn't expect them. For example, this performance problem, which I didn't expect first when I planned the architecture of the tool. So that's one thing I have to address and which takes a lot of time to find the reasons where the problem is exactly. So one more question about the development of the tool. So it's part of your PhD project. So did you develop it all yourself? Or did you also consider what, for instance, I did in my PhD project using students with Bachelor Master's thesis to continue the development of the tool. Maybe it could free up some time for you. I don't know. Maybe at the moment I'm working only myself on that tool and I'm quite certain that I will finish it in the next months. So no, at the moment, no reason to involve other people. Hi, I would be interested how the report looks like if you have all. Do you have some categorizations like also that you can configure contrast level or descriptions availability or reachability by keyboard? Not yet. At the moment it's only structured by the rules. So you get for every rule, you get the output. In later versions you will be able to filter the results, for example, for the BC AG conformance level or by other means. But for example, for contrast, the web content accessibility guidelines give a very explicit measurement, which is necessary. So there's not much interpretation or much room for the initials there. May I also ask a question? Of course. I was surprised that you don't crawl the whole website, but you already need a half an hour for one page. This is a large page, but anyhow, this makes the problem quite complex to increase the speed of your analysis. I think one of the reasons why it takes so long is that the communication between the bot and the service is not efficient yet. And the saving of the results takes too long. But the problem is, I already start multiple threads in the bot to do the evaluation, but you can't start your own threads in a Java web application. So I had to split this. And I think I'm on a good way at the moment, and the development to cut down the time the evaluation needs. And when I'm satisfied with the performance for one page, then I will do the things for the other side, for the whole side. I don't think I will crawl the page site really. I think you will define which pages you want to be evaluated because many sites have some typical pages. They are built by templates. And if you are the developer, you know more or less which pages are typical for your page so that you don't have to evaluate every single page, you do the important ones, the complex ones, and leave out the smaller ones which are identical from their structure but only differ in their content. Okay. Any more questions? No, I don't see. So thank you very much for your talk. Thank you. And thank you for the audience. And then I think we can close the session for the break. And see you in the afternoon.
 So, hello everyone.  We have now the second talk in the science meet, Froskon meet science track and I'm introduced  to you Jens Pelseter from the University of Bremen, right?  And he's right now doing his PhD and the topic he is talking today is part of his, of his  PhD and he talks about the automatic approach for testing the accessibility of web pages.  Truly an important issue, nevertheless maybe one issue not on the top list of the developers.  So it's your stage.  Yeah, thank you.  Okay.  Yeah.  But it describes the thing I want to talk about.  It's best.  First some words about accessibility about web pages.  Most people think about blind people if they hear this topic.  That's not the truth, the complete truth.  Blind people are only one part of accessibility.  An important part nevertheless, but that there are many more impairments which can affect  the way you can use web pages.  One example about 8% of the male population can't see red and green as colors.  So that's not too small amount of people.  And what are accessible web pages?  There are a few standards in this.  The most important one are the web content accessibility guidelines which are available  in the version 2.1 since last year.  They are organized into four principles.  Percivable, operable, understandable and robust.  Which then are split into 13 guidelines and 78 success criteria which describe where the  various requirements for accessible web pages.  There are also some supplemental documents understanding the way content accessibility  guidelines which describes the intent of the success criteria in detail and the techniques  for web content accessibility guidelines describes how you can implement them.  The next thing is not that easy.  But I will come to that later.  There are a few other standards.  For example the authoring tool accessibility guidelines or the accessible rich internet  application standards.  If you are interested in that I will give a talk in C119 this afternoon about these  standards where I will talk in detail about them.  So if you are interested it's at 2 o'clock this afternoon.  The testing and web page accessibility is quite complex task.  It requires manual work at the moment.  Even there are some browser plugins which you can use to test some parts or some aspects  of the accessibility but not everything.  It still requires very manual work and it's very time consuming and the guidelines from  the web content accessibility guidelines are also open for interpretation.  The W3C has recognized this and is currently developing a new standard.  Accessibility conformance testing rules which defines a format for describing accessibility  conformance testing rules.  And first rules using the standards are available at a GitHub repository.  The second is that it's still a textual description so nothing which you can use for automatic  tests.  The intent is to provide a tool implementers a guideline for these things but at the moment  there are not very much tools which implement that.  And the idea which I had is you should automate as much as possible if you test these and  for the steps which require manual work you should guide the users to these steps so that  they know exactly what to check for.  And I developed as part of my PhD tool which is structured like this.  First there is a service which has an ontology with the rules and executes the test using  Selenium.  I don't know if Selenium is, if you know Selenium that's a tool which you can use for testing  web user interfaces.  It's often used to check if a user interface works correctly.  And the service uses Selenium to do that, provides a user interface or the web user  interface.  And the real testing is done by multiple instances of a bot which then executes the Selenium  test and tests the web side.  The web user interface looks at a moment like this, it's all implemented in Java because  the reason for that is that APIs for using ontologies are only available in Java.  So the easy way was to do the rest also in Java.  If you want to evaluate a web page you simply put the URL here, click start and then the  tool starts to run, analyzes the document and executes all tests it knows.  It first determines which rules are applicable for each element and then executes for each  element the applicable test.  And at the end you get some screen like this, this way you see the task which requires manual  work and if you then click on the button you see a screenshot with the element and the  question and simply click past or failed.  And at the end, when all tests are finished, you get an overview about the rules.  For this page many rules are not applicable.  And here you see some rules which have passed or failed and that's it.  The short description of these Act Rules looks like this.  Yeah, so here that's the description when this rule is applicable.  Don't need to read this now.  And in my tool the translation looks like this if you put that in the pseudo code.  The rule has three conditions which the element must match so that the rule is applicable.  And in this case it should be a button, should be exposed to the accessibility tree.  Every browser builds an additional tree of the elements of a web page which is available  for assistive technologies, for example screen readers.  And not all elements are exposed to this tree.  And this element should not be an input element with the type image.  Which yeah.  And now if you are lucky.  Yeah, it still runs.  Before the talk I started evaluation of the FrostCon web page which is still running.  That's also one problem which I'm currently working on.  It takes very, at the moment the testing takes very long because the problem is a web page  has, a normal web page has many elements.  FrostCon web page has not so much but for example the start page of our university has  about 2000 HTML elements.  And at the moment it takes the tool needs very long to test them all.  But I think I will, I can cut this time so that it takes about half an hour for such  a large page then to test.  It's still a long time but if you want to evaluate such a large page manually you will  need longer.  Because you have to look for each element and that for each tool.  So the tool will save you time.  And yeah.  I think the tool also has some problem.  Okay, yeah.  That's so far.  Yeah, that's the current status.  I'm currently working on bug fix and optimizations.  And I'm looking for possible participants which want to evaluate this tool if it's finished.  So if you're interested please write me in mail and I will get back to you when the tool  is ready.  Yeah.  Then some other talks.  I will give two other talks here on the FrostCon one this afternoon in C119 which where I will  talk about the standards for accessible web pages.  And tomorrow I will give a talk where I will explain why accessible web pages are also  useful for everybody.  Not only people with impairments.  And some other advertisements.  I'm also a member of the CMS garden which has also a boot here.  And we have our own unconference in November.  So if you want to meet us and learn about web content management systems you can come  to Essen.  Attendance is free as here.  Yeah.  Okay, questions.  Yeah.  You started the scan from the web page.  So you started the scan from a web page.  Is there also the possibility to start it from command line?  No.  So that would enable.  Not yet.  Okay.  Maybe I will work on that.  What you can do is to use the browser extension.  So you can start it directly from the browser.  But that extension is still in development.  The reason I'm asking is if you could integrate this into your CI pipeline that would be much  more pleasant because you don't have to wait half an hour or however long it takes on your  local computer.  But you can just push to the server, have it build it.  You can leave this web page.  It's not necessary to leave it open.  Yeah, but it still has to be running on my local computer.  No, no, no, no.  It's a web service.  It will run on a server.  On a server.  But if I have that one instance of the page that I want to test and locally I want to  continue development, it's much easier if I have multiple versions.  Yeah.  No, no, that's a second step to integration in CI pipelines.  Okay.  Thank you.  Yeah.  My question is in which different output formats can you get the report?  At the moment, the question was in which out.  Okay.  At the moment, you can only get an output for humans.  When the tool is finished, there is a standard for accessibility evaluation reports in a  JSON format.  Which you can get the report then as export.  So as JSON, maybe other formats also, but the standard will be implemented.  Jason would be quite fine.  Thank you.  Yeah.  Thanks.  What experience did you already get with this tool?  So have you run it yourself on various websites?  Or had many other projects that tried it?  So far the results were that many webpages have some accessibility problems.  Sometimes in parts which you don't expect.  For example, one of the rules is that every element must have an accessible name which  is exposed to this assistive technologies and not much webpages have that for all elements  where it's necessary.  It's not only text for images, but also real labels for input elements.  I've seen many examples where you have a visual label for an input element, but it's not  connected with the input in a way which can be recognized by assistive technology.  Yeah.  My question would be whether you have to specify each page on your website?  At the moment, yes.  That's also one of the next steps to automate the scan for a whole site.  It's now the second step.  First, get it running for a single document and then look into how you can do it for multiple  pages.  Okay, I have one more.  Does it also work with, let's say, JavaScript intensive pages that have only small static  parts and do a lot of rendering based on information that comes back after just some JavaScript  calls?  It should.  That's also one thing which has to be tested, how it works.  Maybe you will have to define some interactions which the tool should do in the background.  I don't know if you know Selenium.  In the background, the bot starts a Firefox box, so JavaScript is supported in this way,  but maybe you will have to define some interactions with the page to get the complete rendering.  Only JavaScript-based web user interfaces are really multiple pages, even if it's one  page.  So that's also one of the next steps if the tool is running fine for normal static web  pages.  Any more questions?  How is it with reliability of the tests?  Do you specify your rules?  Will it get through the cases?  Thankfully, the people which do define these act rules, which I'm currently using as base,  also define test cases for tools which implement them.  So that should be easy to check.  That's also one of the parts which I'm currently working on.  As you maybe know, you find many problems when you start testing and when you are developing  such novel tools.  So sometimes you are not as fast with the development as you want.  I hope to show that I could show more today, but unfortunately there are some nasty bugs  which I have to address in parts where I didn't expect them.  For example, this performance problem, which I didn't expect first when I planned the architecture  of the tool.  So that's one thing I have to address and which takes a lot of time to find the reasons  where the problem is exactly.  So one more question about the development of the tool.  So it's part of your PhD project.  So did you develop it all yourself?  Or did you also consider what, for instance, I did in my PhD project using students with  Bachelor Master's thesis to continue the development of the tool.  Maybe it could free up some time for you.  I don't know.  Maybe at the moment I'm working only myself on that tool and I'm quite certain that I  will finish it in the next months.  So no, at the moment, no reason to involve other people.  Hi, I would be interested how the report looks like if you have all.  Do you have some categorizations like also that you can configure contrast level or descriptions  availability or reachability by keyboard?  Not yet.  At the moment it's only structured by the rules.  So you get for every rule, you get the output.  In later versions you will be able to filter the results, for example, for the BC AG conformance  level or by other means.  But for example, for contrast, the web content accessibility guidelines give a very explicit  measurement, which is necessary.  So there's not much interpretation or much room for the initials there.  May I also ask a question?  Of course.  I was surprised that you don't crawl the whole website, but you already need a half  an hour for one page.  This is a large page, but anyhow, this makes the problem quite complex to increase the  speed of your analysis.  I think one of the reasons why it takes so long is that the communication between the  bot and the service is not efficient yet.  And the saving of the results takes too long.  But the problem is, I already start multiple threads in the bot to do the evaluation, but  you can't start your own threads in a Java web application.  So I had to split this.  And I think I'm on a good way at the moment, and the development to cut down the time the  evaluation needs.  And when I'm satisfied with the performance for one page, then I will do the things for  the other side, for the whole side.  I don't think I will crawl the page site really.  I think you will define which pages you want to be evaluated because many sites have some  typical pages.  They are built by templates.  And if you are the developer, you know more or less which pages are typical for your page  so that you don't have to evaluate every single page, you do the important ones, the complex  ones, and leave out the smaller ones which are identical from their structure but only  differ in their content.  Okay.  Any more questions?  No, I don't see.  So thank you very much for your talk.  Thank you.  And thank you for the audience.  And then I think we can close the session for the break.  And see you in the afternoon.
2019_-_A_semiautomatic_approach_for_testing_the_ac_iY644lrEZxY_0_359.png 2019_-_A_semiautomatic_approach_for_testing_the_ac_iY644lrEZxY_360_443.png 2019_-_A_semiautomatic_approach_for_testing_the_ac_iY644lrEZxY_444_887.png 2019_-_A_semiautomatic_approach_for_testing_the_ac_iY644lrEZxY_888_1067.png 2019_-_A_semiautomatic_approach_for_testing_the_ac_iY644lrEZxY_1068_1271.png 2019_-_A_semiautomatic_approach_for_testing_the_ac_iY644lrEZxY_1272_1763.png 2019_-_A_semiautomatic_approach_for_testing_the_ac_iY644lrEZxY_1764_2855.png 2019_-_A_semiautomatic_approach_for_testing_the_ac_iY644lrEZxY_2856_3011.png 2019_-_A_semiautomatic_approach_for_testing_the_ac_iY644lrEZxY_3012_3131.png 2019_-_A_semiautomatic_approach_for_testing_the_ac_iY644lrEZxY_3132_3359.png 2019_-_A_semiautomatic_approach_for_testing_the_ac_iY644lrEZxY_3360_3407.png 2019_-_A_semiautomatic_approach_for_testing_the_ac_iY644lrEZxY_3408_3479.png 2019_-_A_semiautomatic_approach_for_testing_the_ac_iY644lrEZxY_3480_3563.png 2019_-_A_semiautomatic_approach_for_testing_the_ac_iY644lrEZxY_3564_3815.png 2019_-_A_semiautomatic_approach_for_testing_the_ac_iY644lrEZxY_3816_4583.png 2019_-_A_semiautomatic_approach_for_testing_the_ac_iY644lrEZxY_4584_5435.png 2019_-_A_semiautomatic_approach_for_testing_the_ac_iY644lrEZxY_5436_8807.png 2019_-_A_semiautomatic_approach_for_testing_the_ac_iY644lrEZxY_8808_8891.png 2019_-_A_semiautomatic_approach_for_testing_the_ac_iY644lrEZxY_8892_9275.png 2019_-_A_semiautomatic_approach_for_testing_the_ac_iY644lrEZxY_9276_12371.png 2019_-_A_semiautomatic_approach_for_testing_the_ac_iY644lrEZxY_12372_14435.png 2019_-_A_semiautomatic_approach_for_testing_the_ac_iY644lrEZxY_14436_14915.png 2019_-_A_semiautomatic_approach_for_testing_the_ac_iY644lrEZxY_14916_15551.png 2019_-_A_semiautomatic_approach_for_testing_the_ac_iY644lrEZxY_15552_16127.png 2019_-_A_semiautomatic_approach_for_testing_the_ac_iY644lrEZxY_16128_16187.png 2019_-_A_semiautomatic_approach_for_testing_the_ac_iY644lrEZxY_16188_16511.png 2019_-_A_semiautomatic_approach_for_testing_the_ac_iY644lrEZxY_16512_17663.png 2019_-_A_semiautomatic_approach_for_testing_the_ac_iY644lrEZxY_17664_17711.png 2019_-_A_semiautomatic_approach_for_testing_the_ac_iY644lrEZxY_17712_20363.png 2019_-_A_semiautomatic_approach_for_testing_the_ac_iY644lrEZxY_20364_21143.png 2019_-_A_semiautomatic_approach_for_testing_the_ac_iY644lrEZxY_21144_32819.png 2019_-_A_semiautomatic_approach_for_testing_the_ac_iY644lrEZxY_32820_33575.png 2019_-_A_semiautomatic_approach_for_testing_the_ac_iY644lrEZxY_33576_34691.png 2019_-_A_semiautomatic_approach_for_testing_the_ac_iY644lrEZxY_34692_35027.png 2019_-_A_semiautomatic_approach_for_testing_the_ac_iY644lrEZxY_35028_35435.png 2019_-_A_semiautomatic_approach_for_testing_the_ac_iY644lrEZxY_35436_35579.png 2019_-_A_semiautomatic_approach_for_testing_the_ac_iY644lrEZxY_35580_36419.png 2019_-_A_semiautomatic_approach_for_testing_the_ac_iY644lrEZxY_36420_37019.png 2019_-_A_semiautomatic_approach_for_testing_the_ac_iY644lrEZxY_37020_37859.png 2019_-_A_semiautomatic_approach_for_testing_the_ac_iY644lrEZxY_37860_39491.png 2019_-_A_semiautomatic_approach_for_testing_the_ac_iY644lrEZxY_39492_39599.png 2019_-_A_semiautomatic_approach_for_testing_the_ac_iY644lrEZxY_39600_41171.png 2019_-_A_semiautomatic_approach_for_testing_the_ac_iY644lrEZxY_41172_42539.png 2019_-_A_semiautomatic_approach_for_testing_the_ac_iY644lrEZxY_42540_42935.png 2019_-_A_semiautomatic_approach_for_testing_the_ac_iY644lrEZxY_42936_43545.png
So, hello everyone. We have now the second talk in the science meet, Froskon meet science track and I'm introduced to you Jens Pelseter from the University of Bremen, right? And he's right now doing his PhD and the topic he is talking today is part of his, of his PhD and he talks about the automatic approach for testing the accessibility of web pages. Truly an important issue, nevertheless maybe one issue not on the top list of the developers. So it's your stage. Yeah, thank you. Okay. Yeah. But it describes the thing I want to talk about. It's best. First some words about accessibility about web pages. Most people think about blind people if they hear this topic. That's not the truth, the complete truth. Blind people are only one part of accessibility. An important part nevertheless, but that there are many more impairments which can affect the way you can use web pages. One example about 8% of the male population can't see red and green as colors. So that's not too small amount of people. And what are accessible web pages? There are a few standards in this. The most important one are the web content accessibility guidelines which are available in the version 2.1 since last year. They are organized into four principles. Percivable, operable, understandable and robust. Which then are split into 13 guidelines and 78 success criteria which describe where the various requirements for accessible web pages. There are also some supplemental documents understanding the way content accessibility guidelines which describes the intent of the success criteria in detail and the techniques for web content accessibility guidelines describes how you can implement them. The next thing is not that easy. But I will come to that later. There are a few other standards. For example the authoring tool accessibility guidelines or the accessible rich internet application standards. If you are interested in that I will give a talk in C119 this afternoon about these standards where I will talk in detail about them. So if you are interested it's at 2 o'clock this afternoon. The testing and web page accessibility is quite complex task. It requires manual work at the moment. Even there are some browser plugins which you can use to test some parts or some aspects of the accessibility but not everything. It still requires very manual work and it's very time consuming and the guidelines from the web content accessibility guidelines are also open for interpretation. The W3C has recognized this and is currently developing a new standard. Accessibility conformance testing rules which defines a format for describing accessibility conformance testing rules. And first rules using the standards are available at a GitHub repository. The second is that it's still a textual description so nothing which you can use for automatic tests. The intent is to provide a tool implementers a guideline for these things but at the moment there are not very much tools which implement that. And the idea which I had is you should automate as much as possible if you test these and for the steps which require manual work you should guide the users to these steps so that they know exactly what to check for. And I developed as part of my PhD tool which is structured like this. First there is a service which has an ontology with the rules and executes the test using Selenium. I don't know if Selenium is, if you know Selenium that's a tool which you can use for testing web user interfaces. It's often used to check if a user interface works correctly. And the service uses Selenium to do that, provides a user interface or the web user interface. And the real testing is done by multiple instances of a bot which then executes the Selenium test and tests the web side. The web user interface looks at a moment like this, it's all implemented in Java because the reason for that is that APIs for using ontologies are only available in Java. So the easy way was to do the rest also in Java. If you want to evaluate a web page you simply put the URL here, click start and then the tool starts to run, analyzes the document and executes all tests it knows. It first determines which rules are applicable for each element and then executes for each element the applicable test. And at the end you get some screen like this, this way you see the task which requires manual work and if you then click on the button you see a screenshot with the element and the question and simply click past or failed. And at the end, when all tests are finished, you get an overview about the rules. For this page many rules are not applicable. And here you see some rules which have passed or failed and that's it. The short description of these Act Rules looks like this. Yeah, so here that's the description when this rule is applicable. Don't need to read this now. And in my tool the translation looks like this if you put that in the pseudo code. The rule has three conditions which the element must match so that the rule is applicable. And in this case it should be a button, should be exposed to the accessibility tree. Every browser builds an additional tree of the elements of a web page which is available for assistive technologies, for example screen readers. And not all elements are exposed to this tree. And this element should not be an input element with the type image. Which yeah. And now if you are lucky. Yeah, it still runs. Before the talk I started evaluation of the FrostCon web page which is still running. That's also one problem which I'm currently working on. It takes very, at the moment the testing takes very long because the problem is a web page has, a normal web page has many elements. FrostCon web page has not so much but for example the start page of our university has about 2000 HTML elements. And at the moment it takes the tool needs very long to test them all. But I think I will, I can cut this time so that it takes about half an hour for such a large page then to test. It's still a long time but if you want to evaluate such a large page manually you will need longer. Because you have to look for each element and that for each tool. So the tool will save you time. And yeah. I think the tool also has some problem. Okay, yeah. That's so far. Yeah, that's the current status. I'm currently working on bug fix and optimizations. And I'm looking for possible participants which want to evaluate this tool if it's finished. So if you're interested please write me in mail and I will get back to you when the tool is ready. Yeah. Then some other talks. I will give two other talks here on the FrostCon one this afternoon in C119 which where I will talk about the standards for accessible web pages. And tomorrow I will give a talk where I will explain why accessible web pages are also useful for everybody. Not only people with impairments. And some other advertisements. I'm also a member of the CMS garden which has also a boot here. And we have our own unconference in November. So if you want to meet us and learn about web content management systems you can come to Essen. Attendance is free as here. Yeah. Okay, questions. Yeah. You started the scan from the web page. So you started the scan from a web page. Is there also the possibility to start it from command line? No. So that would enable. Not yet. Okay. Maybe I will work on that. What you can do is to use the browser extension. So you can start it directly from the browser. But that extension is still in development. The reason I'm asking is if you could integrate this into your CI pipeline that would be much more pleasant because you don't have to wait half an hour or however long it takes on your local computer. But you can just push to the server, have it build it. You can leave this web page. It's not necessary to leave it open. Yeah, but it still has to be running on my local computer. No, no, no, no. It's a web service. It will run on a server. On a server. But if I have that one instance of the page that I want to test and locally I want to continue development, it's much easier if I have multiple versions. Yeah. No, no, that's a second step to integration in CI pipelines. Okay. Thank you. Yeah. My question is in which different output formats can you get the report? At the moment, the question was in which out. Okay. At the moment, you can only get an output for humans. When the tool is finished, there is a standard for accessibility evaluation reports in a JSON format. Which you can get the report then as export. So as JSON, maybe other formats also, but the standard will be implemented. Jason would be quite fine. Thank you. Yeah. Thanks. What experience did you already get with this tool? So have you run it yourself on various websites? Or had many other projects that tried it? So far the results were that many webpages have some accessibility problems. Sometimes in parts which you don't expect. For example, one of the rules is that every element must have an accessible name which is exposed to this assistive technologies and not much webpages have that for all elements where it's necessary. It's not only text for images, but also real labels for input elements. I've seen many examples where you have a visual label for an input element, but it's not connected with the input in a way which can be recognized by assistive technology. Yeah. My question would be whether you have to specify each page on your website? At the moment, yes. That's also one of the next steps to automate the scan for a whole site. It's now the second step. First, get it running for a single document and then look into how you can do it for multiple pages. Okay, I have one more. Does it also work with, let's say, JavaScript intensive pages that have only small static parts and do a lot of rendering based on information that comes back after just some JavaScript calls? It should. That's also one thing which has to be tested, how it works. Maybe you will have to define some interactions which the tool should do in the background. I don't know if you know Selenium. In the background, the bot starts a Firefox box, so JavaScript is supported in this way, but maybe you will have to define some interactions with the page to get the complete rendering. Only JavaScript-based web user interfaces are really multiple pages, even if it's one page. So that's also one of the next steps if the tool is running fine for normal static web pages. Any more questions? How is it with reliability of the tests? Do you specify your rules? Will it get through the cases? Thankfully, the people which do define these act rules, which I'm currently using as base, also define test cases for tools which implement them. So that should be easy to check. That's also one of the parts which I'm currently working on. As you maybe know, you find many problems when you start testing and when you are developing such novel tools. So sometimes you are not as fast with the development as you want. I hope to show that I could show more today, but unfortunately there are some nasty bugs which I have to address in parts where I didn't expect them. For example, this performance problem, which I didn't expect first when I planned the architecture of the tool. So that's one thing I have to address and which takes a lot of time to find the reasons where the problem is exactly. So one more question about the development of the tool. So it's part of your PhD project. So did you develop it all yourself? Or did you also consider what, for instance, I did in my PhD project using students with Bachelor Master's thesis to continue the development of the tool. Maybe it could free up some time for you. I don't know. Maybe at the moment I'm working only myself on that tool and I'm quite certain that I will finish it in the next months. So no, at the moment, no reason to involve other people. Hi, I would be interested how the report looks like if you have all. Do you have some categorizations like also that you can configure contrast level or descriptions availability or reachability by keyboard? Not yet. At the moment it's only structured by the rules. So you get for every rule, you get the output. In later versions you will be able to filter the results, for example, for the BC AG conformance level or by other means. But for example, for contrast, the web content accessibility guidelines give a very explicit measurement, which is necessary. So there's not much interpretation or much room for the initials there. May I also ask a question? Of course. I was surprised that you don't crawl the whole website, but you already need a half an hour for one page. This is a large page, but anyhow, this makes the problem quite complex to increase the speed of your analysis. I think one of the reasons why it takes so long is that the communication between the bot and the service is not efficient yet. And the saving of the results takes too long. But the problem is, I already start multiple threads in the bot to do the evaluation, but you can't start your own threads in a Java web application. So I had to split this. And I think I'm on a good way at the moment, and the development to cut down the time the evaluation needs. And when I'm satisfied with the performance for one page, then I will do the things for the other side, for the whole side. I don't think I will crawl the page site really. I think you will define which pages you want to be evaluated because many sites have some typical pages. They are built by templates. And if you are the developer, you know more or less which pages are typical for your page so that you don't have to evaluate every single page, you do the important ones, the complex ones, and leave out the smaller ones which are identical from their structure but only differ in their content. Okay. Any more questions? No, I don't see. So thank you very much for your talk. Thank you. And thank you for the audience. And then I think we can close the session for the break. And see you in the afternoon.
automatic approach for testing the accessibility you can use web pages. One example about 8% of requirements for accessible web pages. There are accessibility guidelines describes how you can testing and web page accessibility is quite consuming and the guidelines from the web content accessibility guidelines are also open for interpretation. The W3C has recognized this and is currently developing a new standard. Accessibility conformance testing rules which defines a format for describing accessibility conformance testing rules. And first rules using test these and for the steps which require manual developed as part of my PhD tool which is the test using Selenium. I don't know if Selenium screen like this, this way you see the task which requires manual work and if you then click on the an overview about the rules. For this page many additional tree of the elements of a web page which where I will talk about the standards for accessible web pages. And tomorrow I will give a the standard will be implemented. Jason would be connected with the input in a way which can be background. I don't know if you know Selenium. In you will have to define some interactions with thesis to continue the development of the tool. hour for one page. This is a large page, but are the developer, you know more or less which