Actionaz: Automation for everyone
Mercier-Ganady, Jonathan
Actionaz is a task automation tool. It allows you to create and execute action lists. You don't need to known any programming language to use it: its intuitive interface allows you to create action lists (scripts) using drag & drop. This event will introduce the features of the program and what can be done to improve it. Advanced users can use JavaScript (EcmaScript) to extend its functionality.Actionaz is free software and runs under Windows and GNU/Linux. Binaries are available for Windows and Ubuntu for both 32 and 64 bits. It uses the Qt framework.
 So, hello everyone. So the topic of this talk would be Action S3, automation for everyone. So first, what do we want to, what is actually the meaning of automation because that's a word that's used in many contexts. In that case, it's software automation, it's a software automation tool. It means that it's most often used in games, software validation, business applications. So it's not like software that you can use to test your program. You can do that, but it has a wider, broader range of users. So it works as scripts. And scripts contain different actions and actions can be house clicks, key presses, window management actions, file access, etc. And there are 46 different actions. So for everyone, because you don't need to learn any new programming language to use it, you just have to drag and drop the actions. And that allows non-programmers to use that tool and automate some actions. But it's still developer friendly because you can program, you can enter some JavaScript code in the parameters of the actions. And that JavaScript code will be evaluated when the action is executed. So it allows you more customization and you also can write an entire file in JavaScript using the classes that are provided by Actioners. So you can really write only in JavaScript if you want to. So let's see a short example. Here I have a script. And on the left side, we can see this is the main, of course, the main window of Actioners. And you can see on the left, the different actions that are available. And the center, you can see the list of actions, the script that I've added here. And with a line number that you can write, you can write a label if you want to, name of the actions and some comments if you want to explain what this action does. And on the right side, you have a console where there are execution errors will be printed and you can also write some text there if you want to debug. So let's see the first action. It's a variable action. Here we will create a variable named firstDemUrl and it's type string. And here you can see that there is a small red triangle. It means that the text I have entered here is not regular text, it's the JavaScript. So here in that case, I create a simple string and I use a JavaScript function to get the current year. And that all of that builds a string that contains the URL to this lightning talk. So the goal of the whole script, I don't think I've said it, is to show, to open a browser, Firefox, and to show the page of this lightning talk. So the next action is a detached command because you can run, of course, commands and in that case, we run the command Firefox and we use a parameter here. And you can see that here it's not JavaScript, it's regular text, but you can insert in regular text a variable. Here it's firstDemUrl and it starts with a dollar like in PHP. So it allows people who don't want to, who don't know how to program in JavaScript to insert some basic variables in text. Then we have a window condition. The idea of that is to wait for a window with a specific title to appear or there are, of course, other conditions, but in that case, we have the title is firstDem and you can see there is a star before and after because you can use some wild cards if you want to have a title in that case, which was the title contains firstDem. So we wait until it exists because if it's true, that means that we found it, we do nothing, that means that we go to the next action. And if it doesn't exist, then we wait. The script execution will block on this action until we found that window. And then we have a pause, of course, we make the pause, that's amazing, for here for one second. So that's cool. The next action is more interesting. It's findImage action. Here this allows you to search for an image either on the screen, within a screenshot or within another action or within a window. That uses some openCV algorithm and here we search on the screen for an image called firstDem and it's image here. So it's the firstDem logo because we need to search that on the browser and we store the results and the position we found in a new variable called firstDem position. Then there are four other actions. I will not detail each one of them because it would take some time. But the idea here is to move the cursor on the position we found and then scroll the wheel to scroll to get down on the page and find the action as text here and then click on it. And that works. It should work. So yeah, a bit of history. I started coding action as in 2005 because a friend wanted to play a game where he had to click often on something and well, it was written in C plus. That's not the typo. It's really C plus because it was horrible. It was some kind of strange mix between C and C plus plus. So it's a freeware. So it's not available because it's so horrible. It used the MFC. So in 2006 I rewrote the whole stuff and I used WX widgets which was quite cool and it was still freeware. So in 2009 I rewrote for the second time the whole program. Now it's in C plus plus and it used Qt. So now it also works on Linux. Okay. So on platforms and license, it works on GNU Linux-based distributions. There is a package in the Debian repository. So it's easy to install and Debian, Wintu, Mint, et cetera. It should work without any problems on other distributions but I don't have any package for them. So if someone is interested, of course I will be very interested to have packages for these platforms, of course. It also works on Windows, XP and newer and it's free software. It's released under the GNU GPL license. So now some architectural stuff. There are two programs. One is called Actionas. It's the main program with a window where you can edit your scripts and add some actions and also you can start them. But there is also a second program called Actexec which is a small program without any interface that allows you to execute a script. It's used also on Windows because on Windows you can make some SFX scripts which is actually 7-zip archive that is automatically, that is auto-extractable so you can run a script on a computer where Actionas is not installed. I haven't done that for Linux because I don't really see the interest in that and it would be a bit more complicated and I don't know if there is a really need for that but that could be done, of course. Actions are in packs and each pack is a plugin. So it's easy to write new actions. Each action could be a plugin but the idea of the packs would have to share some dependencies between the actions in one file. That was the main idea. 90% of the code is platform independent because it's cute so it's really not a problem to put it to a new platform. The 10% are of course some specific stuff like emulating a click and something like that which don't exist on Qt. So that depends of course of the operating system but it's a code which is not really separated from the rest. That's really a problem but I have a plan to put that somewhere which could be more easier to put that to new platforms. So of course help is needed because there are lots of new ideas, lots of improvements, some bug fixes that could be done. For instance, for the new ideas there is a script resources idea and the basic idea of that is to take a file which is script dependent. For instance, an image that you want to find and to encode that in base 64 and include that in the XML of the script file. I have already done some coding for that but it's not finished yet. The script recording, the idea of that is to allow to have a button when you can click and it will record the actions you do and automatically add them. Mainly works for clicks and keyboard actions. The problem with that is that there is no cross-platform way to store a key. That's really complicated because each platform has its own ways of storing them and defining them. So that's actually more tricky than it seems. There is also an idea to wait for input. The idea of that is to say we want to wait for the user to click or to press any key and it should work even if action has, of course, doesn't have the focus. So on Windows, it works, for instance, it works by making a hook. The sound recording, that's obvious. A task planner, the idea of that is to have some program running all the time and waiting for some conditions to be fulfilled and execute one or more scripts. And of course, the next idea is to be to parallelize execution of scripts. Because, yeah, and that, of course, asks some, there are some issues with that because what happens if one script want to move the mouse to one position, the mouse cursor, of course, that's the mouse itself. And anyway, another one wants to do the same. What happens then? So that could be a problem that has to be solved. And the other idea is to make a script database that allows users to upload scripts and download scripts from others. Of course, there will have to be some checking for the scripts because a script can potentially do anything on your computer. So it's better if it's not some evil stuff. And other things, web actions, the idea would be to execute some actions on a web page using JavaScript, there are, of course, more specific software doing that. And of course, probably much better. But the idea is to have some basic stuff to do that. Qt 5, of course, would be a good idea. Usability improvements, documentation, there is already some reference documentation, but tutorials would be a good idea. Packaging, there is a packaging for Debian, but it would be nice to have some packaging for other distributions. Macintosh would be a good idea, I don't have one, so if someone wants to, of course, it's really welcome. If you need more information, there is a website. There are three different mailing lists, one for developers, two for users. The source code is available on GitHub. Yeah, that's everything. So thank you, everyone. Thank you for them, of course. And if you have some questions, that will be outside the user. Thank you. Thank you.
 So, hello everyone.  So the topic of this talk would be Action S3, automation for everyone.  So first, what do we want to, what is actually the meaning of automation because that's a  word that's used in many contexts.  In that case, it's software automation, it's a software automation tool.  It means that it's most often used in games, software validation, business applications.  So it's not like software that you can use to test your program.  You can do that, but it has a wider, broader range of users.  So it works as scripts.  And scripts contain different actions and actions can be house clicks, key presses,  window management actions, file access, etc.  And there are 46 different actions.  So for everyone, because you don't need to learn any new programming language to use  it, you just have to drag and drop the actions.  And that allows non-programmers to use that tool and automate some actions.  But it's still developer friendly because you can program, you can enter some JavaScript  code in the parameters of the actions.  And that JavaScript code will be evaluated when the action is executed.  So it allows you more customization and you also can write an entire file in JavaScript  using the classes that are provided by Actioners.  So you can really write only in JavaScript if you want to.  So let's see a short example.  Here I have a script.  And on the left side, we can see this is the main, of course, the main window of Actioners.  And you can see on the left, the different actions that are available.  And the center, you can see the list of actions, the script that I've added here.  And with a line number that you can write, you can write a label if you want to, name  of the actions and some comments if you want to explain what this action does.  And on the right side, you have a console where there are execution errors will be printed  and you can also write some text there if you want to debug.  So let's see the first action.  It's a variable action.  Here we will create a variable named firstDemUrl and it's type string.  And here you can see that there is a small red triangle.  It means that the text I have entered here is not regular text, it's the JavaScript.  So here in that case, I create a simple string and I use a JavaScript function to get the  current year.  And that all of that builds a string that contains the URL to this lightning talk.  So the goal of the whole script, I don't think I've said it, is to show, to open a browser,  Firefox, and to show the page of this lightning talk.  So the next action is a detached command because you can run, of course, commands and in that  case, we run the command Firefox and we use a parameter here.  And you can see that here it's not JavaScript, it's regular text, but you can insert in regular  text a variable.  Here it's firstDemUrl and it starts with a dollar like in PHP.  So it allows people who don't want to, who don't know how to program in JavaScript to  insert some basic variables in text.  Then we have a window condition.  The idea of that is to wait for a window with a specific title to appear or there are, of  course, other conditions, but in that case, we have the title is firstDem and you can  see there is a star before and after because you can use some wild cards if you want to  have a title in that case, which was the title contains firstDem.  So we wait until it exists because if it's true, that means that we found it, we do nothing,  that means that we go to the next action.  And if it doesn't exist, then we wait.  The script execution will block on this action until we found that window.  And then we have a pause, of course, we make the pause, that's amazing, for here for one  second.  So that's cool.  The next action is more interesting.  It's findImage action.  Here this allows you to search for an image either on the screen, within a screenshot  or within another action or within a window.  That uses some openCV algorithm and here we search on the screen for an image called firstDem  and it's image here.  So it's the firstDem logo because we need to search that on the browser and we store  the results and the position we found in a new variable called firstDem position.  Then there are four other actions.  I will not detail each one of them because it would take some time.  But the idea here is to move the cursor on the position we found and then scroll the wheel  to scroll to get down on the page and find the action as text here and then click on it.  And that works.  It should work.  So yeah, a bit of history.  I started coding action as in 2005 because a friend wanted to play a game where he had  to click often on something and well, it was written in C plus.  That's not the typo.  It's really C plus because it was horrible.  It was some kind of strange mix between C and C plus plus.  So it's a freeware.  So it's not available because it's so horrible.  It used the MFC.  So in 2006 I rewrote the whole stuff and I used WX widgets which was quite cool and  it was still freeware.  So in 2009 I rewrote for the second time the whole program.  Now it's in C plus plus and it used Qt.  So now it also works on Linux.  Okay.  So on platforms and license, it works on GNU Linux-based distributions.  There is a package in the Debian repository.  So it's easy to install and Debian, Wintu, Mint, et cetera.  It should work without any problems on other distributions but I don't have any package  for them.  So if someone is interested, of course I will be very interested to have packages for these  platforms, of course.  It also works on Windows, XP and newer and it's free software.  It's released under the GNU GPL license.  So now some architectural stuff.  There are two programs.  One is called Actionas.  It's the main program with a window where you can edit your scripts and add some actions  and also you can start them.  But there is also a second program called Actexec which is a small program without any interface  that allows you to execute a script.  It's used also on Windows because on Windows you can make some SFX scripts which is actually  7-zip archive that is automatically, that is auto-extractable so you can run a script  on a computer where Actionas is not installed.  I haven't done that for Linux because I don't really see the interest in that and it would  be a bit more complicated and I don't know if there is a really need for that but that  could be done, of course.  Actions are in packs and each pack is a plugin.  So it's easy to write new actions.  Each action could be a plugin but the idea of the packs would have to share some dependencies  between the actions in one file.  That was the main idea.  90% of the code is platform independent because it's cute so it's really not a problem to  put it to a new platform.  The 10% are of course some specific stuff like emulating a click and something like that  which don't exist on Qt.  So that depends of course of the operating system but it's a code which is not really  separated from the rest.  That's really a problem but I have a plan to put that somewhere which could be more  easier to put that to new platforms.  So of course help is needed because there are lots of new ideas, lots of improvements,  some bug fixes that could be done.  For instance, for the new ideas there is a script resources idea and the basic idea  of that is to take a file which is script dependent.  For instance, an image that you want to find and to encode that in base 64 and include  that in the XML of the script file.  I have already done some coding for that but it's not finished yet.  The script recording, the idea of that is to allow to have a button when you can click  and it will record the actions you do and automatically add them.  Mainly works for clicks and keyboard actions.  The problem with that is that there is no cross-platform way to store a key.  That's really complicated because each platform has its own ways of storing them and defining  them.  So that's actually more tricky than it seems.  There is also an idea to wait for input.  The idea of that is to say we want to wait for the user to click or to press any key  and it should work even if action has, of course, doesn't have the focus.  So on Windows, it works, for instance, it works by making a hook.  The sound recording, that's obvious.  A task planner, the idea of that is to have some program running all the time and waiting  for some conditions to be fulfilled and execute one or more scripts.  And of course, the next idea is to be to parallelize execution of scripts.  Because, yeah, and that, of course, asks some, there are some issues with that because what  happens if one script want to move the mouse to one position, the mouse cursor, of course,  that's the mouse itself.  And anyway, another one wants to do the same.  What happens then?  So that could be a problem that has to be solved.  And the other idea is to make a script database that allows users to upload scripts and download  scripts from others.  Of course, there will have to be some checking for the scripts because a script can potentially  do anything on your computer.  So it's better if it's not some evil stuff.  And other things, web actions, the idea would be to execute some actions on a web page using  JavaScript, there are, of course, more specific software doing that.  And of course, probably much better.  But the idea is to have some basic stuff to do that.  Qt 5, of course, would be a good idea.  Usability improvements, documentation, there is already some reference documentation, but  tutorials would be a good idea.  Packaging, there is a packaging for Debian, but it would be nice to have some packaging  for other distributions.  Macintosh would be a good idea, I don't have one, so if someone wants to, of course, it's  really welcome.  If you need more information, there is a website.  There are three different mailing lists, one for developers, two for users.  The source code is available on GitHub.  Yeah, that's everything.  So thank you, everyone.  Thank you for them, of course.  And if you have some questions, that will be outside the user.  Thank you.  Thank you.
Actionaz_Automation_for_everyone_Ofdoug0Otxc_2_0_275.png Actionaz_Automation_for_everyone_Ofdoug0Otxc_2_276_347.png Actionaz_Automation_for_everyone_Ofdoug0Otxc_2_348_1511.png Actionaz_Automation_for_everyone_Ofdoug0Otxc_2_1512_1799.png Actionaz_Automation_for_everyone_Ofdoug0Otxc_2_1800_1919.png Actionaz_Automation_for_everyone_Ofdoug0Otxc_2_1920_6395.png Actionaz_Automation_for_everyone_Ofdoug0Otxc_2_6396_10067.png Actionaz_Automation_for_everyone_Ofdoug0Otxc_2_10068_11015.png Actionaz_Automation_for_everyone_Ofdoug0Otxc_2_11016_11147.png Actionaz_Automation_for_everyone_Ofdoug0Otxc_2_11148_11267.png Actionaz_Automation_for_everyone_Ofdoug0Otxc_2_11268_12167.png Actionaz_Automation_for_everyone_Ofdoug0Otxc_2_12168_13067.png Actionaz_Automation_for_everyone_Ofdoug0Otxc_2_13068_13319.png Actionaz_Automation_for_everyone_Ofdoug0Otxc_2_13320_14375.png Actionaz_Automation_for_everyone_Ofdoug0Otxc_2_14376_14591.png Actionaz_Automation_for_everyone_Ofdoug0Otxc_2_14592_15251.png Actionaz_Automation_for_everyone_Ofdoug0Otxc_2_15252_15347.png Actionaz_Automation_for_everyone_Ofdoug0Otxc_2_15348_17399.png Actionaz_Automation_for_everyone_Ofdoug0Otxc_2_17400_20147.png Actionaz_Automation_for_everyone_Ofdoug0Otxc_2_20148_20591.png Actionaz_Automation_for_everyone_Ofdoug0Otxc_2_20592_20652.png
So, hello everyone. So the topic of this talk would be Action S3, automation for everyone. So first, what do we want to, what is actually the meaning of automation because that's a word that's used in many contexts. In that case, it's software automation, it's a software automation tool. It means that it's most often used in games, software validation, business applications. So it's not like software that you can use to test your program. You can do that, but it has a wider, broader range of users. So it works as scripts. And scripts contain different actions and actions can be house clicks, key presses, window management actions, file access, etc. And there are 46 different actions. So for everyone, because you don't need to learn any new programming language to use it, you just have to drag and drop the actions. And that allows non-programmers to use that tool and automate some actions. But it's still developer friendly because you can program, you can enter some JavaScript code in the parameters of the actions. And that JavaScript code will be evaluated when the action is executed. So it allows you more customization and you also can write an entire file in JavaScript using the classes that are provided by Actioners. So you can really write only in JavaScript if you want to. So let's see a short example. Here I have a script. And on the left side, we can see this is the main, of course, the main window of Actioners. And you can see on the left, the different actions that are available. And the center, you can see the list of actions, the script that I've added here. And with a line number that you can write, you can write a label if you want to, name of the actions and some comments if you want to explain what this action does. And on the right side, you have a console where there are execution errors will be printed and you can also write some text there if you want to debug. So let's see the first action. It's a variable action. Here we will create a variable named firstDemUrl and it's type string. And here you can see that there is a small red triangle. It means that the text I have entered here is not regular text, it's the JavaScript. So here in that case, I create a simple string and I use a JavaScript function to get the current year. And that all of that builds a string that contains the URL to this lightning talk. So the goal of the whole script, I don't think I've said it, is to show, to open a browser, Firefox, and to show the page of this lightning talk. So the next action is a detached command because you can run, of course, commands and in that case, we run the command Firefox and we use a parameter here. And you can see that here it's not JavaScript, it's regular text, but you can insert in regular text a variable. Here it's firstDemUrl and it starts with a dollar like in PHP. So it allows people who don't want to, who don't know how to program in JavaScript to insert some basic variables in text. Then we have a window condition. The idea of that is to wait for a window with a specific title to appear or there are, of course, other conditions, but in that case, we have the title is firstDem and you can see there is a star before and after because you can use some wild cards if you want to have a title in that case, which was the title contains firstDem. So we wait until it exists because if it's true, that means that we found it, we do nothing, that means that we go to the next action. And if it doesn't exist, then we wait. The script execution will block on this action until we found that window. And then we have a pause, of course, we make the pause, that's amazing, for here for one second. So that's cool. The next action is more interesting. It's findImage action. Here this allows you to search for an image either on the screen, within a screenshot or within another action or within a window. That uses some openCV algorithm and here we search on the screen for an image called firstDem and it's image here. So it's the firstDem logo because we need to search that on the browser and we store the results and the position we found in a new variable called firstDem position. Then there are four other actions. I will not detail each one of them because it would take some time. But the idea here is to move the cursor on the position we found and then scroll the wheel to scroll to get down on the page and find the action as text here and then click on it. And that works. It should work. So yeah, a bit of history. I started coding action as in 2005 because a friend wanted to play a game where he had to click often on something and well, it was written in C plus. That's not the typo. It's really C plus because it was horrible. It was some kind of strange mix between C and C plus plus. So it's a freeware. So it's not available because it's so horrible. It used the MFC. So in 2006 I rewrote the whole stuff and I used WX widgets which was quite cool and it was still freeware. So in 2009 I rewrote for the second time the whole program. Now it's in C plus plus and it used Qt. So now it also works on Linux. Okay. So on platforms and license, it works on GNU Linux-based distributions. There is a package in the Debian repository. So it's easy to install and Debian, Wintu, Mint, et cetera. It should work without any problems on other distributions but I don't have any package for them. So if someone is interested, of course I will be very interested to have packages for these platforms, of course. It also works on Windows, XP and newer and it's free software. It's released under the GNU GPL license. So now some architectural stuff. There are two programs. One is called Actionas. It's the main program with a window where you can edit your scripts and add some actions and also you can start them. But there is also a second program called Actexec which is a small program without any interface that allows you to execute a script. It's used also on Windows because on Windows you can make some SFX scripts which is actually 7-zip archive that is automatically, that is auto-extractable so you can run a script on a computer where Actionas is not installed. I haven't done that for Linux because I don't really see the interest in that and it would be a bit more complicated and I don't know if there is a really need for that but that could be done, of course. Actions are in packs and each pack is a plugin. So it's easy to write new actions. Each action could be a plugin but the idea of the packs would have to share some dependencies between the actions in one file. That was the main idea. 90% of the code is platform independent because it's cute so it's really not a problem to put it to a new platform. The 10% are of course some specific stuff like emulating a click and something like that which don't exist on Qt. So that depends of course of the operating system but it's a code which is not really separated from the rest. That's really a problem but I have a plan to put that somewhere which could be more easier to put that to new platforms. So of course help is needed because there are lots of new ideas, lots of improvements, some bug fixes that could be done. For instance, for the new ideas there is a script resources idea and the basic idea of that is to take a file which is script dependent. For instance, an image that you want to find and to encode that in base 64 and include that in the XML of the script file. I have already done some coding for that but it's not finished yet. The script recording, the idea of that is to allow to have a button when you can click and it will record the actions you do and automatically add them. Mainly works for clicks and keyboard actions. The problem with that is that there is no cross-platform way to store a key. That's really complicated because each platform has its own ways of storing them and defining them. So that's actually more tricky than it seems. There is also an idea to wait for input. The idea of that is to say we want to wait for the user to click or to press any key and it should work even if action has, of course, doesn't have the focus. So on Windows, it works, for instance, it works by making a hook. The sound recording, that's obvious. A task planner, the idea of that is to have some program running all the time and waiting for some conditions to be fulfilled and execute one or more scripts. And of course, the next idea is to be to parallelize execution of scripts. Because, yeah, and that, of course, asks some, there are some issues with that because what happens if one script want to move the mouse to one position, the mouse cursor, of course, that's the mouse itself. And anyway, another one wants to do the same. What happens then? So that could be a problem that has to be solved. And the other idea is to make a script database that allows users to upload scripts and download scripts from others. Of course, there will have to be some checking for the scripts because a script can potentially do anything on your computer. So it's better if it's not some evil stuff. And other things, web actions, the idea would be to execute some actions on a web page using JavaScript, there are, of course, more specific software doing that. And of course, probably much better. But the idea is to have some basic stuff to do that. Qt 5, of course, would be a good idea. Usability improvements, documentation, there is already some reference documentation, but tutorials would be a good idea. Packaging, there is a packaging for Debian, but it would be nice to have some packaging for other distributions. Macintosh would be a good idea, I don't have one, so if someone wants to, of course, it's really welcome. If you need more information, there is a website. There are three different mailing lists, one for developers, two for users. The source code is available on GitHub. Yeah, that's everything. So thank you, everyone. Thank you for them, of course. And if you have some questions, that will be outside the user. Thank you. Thank you.
software automation, it's a software automation tool. It means that it's most often used in can use to test your program. You can do that, everyone, because you don't need to learn any new programming language to use it, you just have to drag and drop the actions. And that allows Windows, XP and newer and it's free software. which is a small program without any interface that allows you to execute a script. It's used that but that could be done, of course. Actions using JavaScript, there are, of course, more for users. The source code is available on