Getting OwnCloud Insights
Stingl, Michael
Where can I as a ownCloud admin get insights to investigate issues with my setup or with ownCloud server or the clients? - owncloud.log - webserver access log - desktop client logs - logs from Android *-debug.apk - Live-logging the iOS client with Xcode - Capturing HTTP(S) requests with mitmproxy
 I come to my talk. Getting some insight in all the parts or the components of the platform from on cloud server and the client. I'm Michael. I work in the customer support team. I also work with the branding service. You will find him on GitHub. Michael Stingel. You will find me on Twitter. Michael Stingel. Elsewhere. You can guess. I sit down and collected all the stuff that could help to find issues with your own cloud, with your clients. So if something isn't working in your setup, it's not always related to own cloud. Sometimes it's a configuration issue in your web server or something else. I found quite a long list of places where you could get information and for me as working customer support, also sometimes we cannot reproduce the issue. Users are reporting, also engineering. Then we need your help providing us the information, send us all the stuff that you can get so to make it possible to help you. We work with engineering to find those issues and fix it if necessary. We will go through all of them. What we have is our own cloud log. If you run the own cloud server, there is a default in your data director. There is a bad example. You shouldn't place your data director inside the HG Docs folder. But this is a default installation. There inside there is a own cloud.log. This is very easy. On a basic technical level, you just go to the admin page, scroll down and there you can see the latest log entries. You can expand this view to see more of them. You can just download the log file and you can configure the logging level. What should be included in the log file. You have a pretty big own cloud instance and you enable high level of logging. It may slow down your own cloud a little bit but we have something for you later. Going a little bit deeper, also for the own cloud log, there is something, some parts that we configure. With the log type, log type is own cloud normally. In the config.php file, we can define where should the log file be placed. Perhaps I need another location. I have the log level, which is three at the moment, which is default. Level zero is everything. With a log date format, you know, date, it's a little bit different sometimes. Some people would like to have their own cloud, not this file, instead log type syslog, sending it to the syslog demon. As I said, if you enable logging for every action on the own cloud server, it may slow your instance down. Again, there's a thing, not many people know, it's a log condition. You can place this in your config.php file and enabling logging only for some users. This user set here will become log level zero, so everything, debug information. Or if you want to investigate issues with a specific app, you can enable logging for specific application. For further debugging, you can add shared secrets to your requests and to only investigate those requests and get the full output in the log. So, the next thing. You need to add it in the HTTP request. The question was about the shared secret. It needs to be set in the HTTP request, so this is not that easy to add this to your client if you don't build it yourself. This is more complex on the technical side. Using this for a specific user app is very easy. Shared secret, this is more like debugging for development. Does it answer your question? More or less. But limiting this to a specific user, it prevents all the issues about slowing your instance down. What other logs do we have? We have the web server log, like example in Apache. On the top, webdav is HTTP. Everything we do, we will find the web server log. For example, this is a little bit of the Apache log. I see the IP address, user name of the client user. I downloaded my hacker songs and seeing also I'm using the latest version of the desktop client on my Mac. Then we also have the time stamps and sometimes we have the request triggering the action of the Apache server, but we don't have the own cloud log. We can compare the times and see something is happening there. We have got all this information. This is just like an outlook. Analyzing the logs, like Horgan mentioned earlier this day, there's for example the Elk stack, and now it's got the Elastic stack. This is a tool that could analyze huge amounts of log files. You could define filters and visualizations to see how often, on which time, from which user, from which location, from which type of client, some incidents happen. This is nothing we provide at the moment, but I would recommend to check this out to see better what's going on in your own cloud systems. I know some of you are using this. Let's go to the client side. In the desktop client, there's an easy way to do this. Just open the client setting, press F12 key, this window pops up, and you can search in it. You can clear again to remove all the previous stuff, and you can save it as a text file, and then you have other information. The problem is something happened. Then I start logging, but I don't have the incident itself. We have a few options to start the client with logging features. The one is adding the parameter log file at the end, and defining the location of the log file, and you can get, at the moment of the start, a list of all the actions that happens in the client. This could become a little bit bigger. My favorite is the log tier option, where you define a directory, and in this directory, you will get a bunch of files. Every Synchron produces one file, and the next Synchron produces a new file, which is, for me, easier to understand. You can set the log expire in hours. If you want to run this on a user's computer, you just say, okay, keep it for 48 hours, and it don't blow your disk. My favorite is defining the own Cloud Sync directory itself as location for the log directory, so the client logs get transmitted to me as an administrator of the instance, and then I could check what's going on in this machine. Yeah. This comes with a desktop sync client on Android. It's a little bit different. It's pretty easy for the Android builds generated in ownPrander. I showed you ownPrander before. If you trigger a build in ownPrander, you get your APK, and every ownPrander build has a twin, and in this debug.apk, there's debugging enabled. Just ask the user to run this APK instead, and later get this log file from the Android device in the file system and let it send to you or send it to ownCloud. This is also possible to connect your device to Android Studio, but that's a little bit more complicated. The application itself needs to be debuggable. There's a manifest file inside there. This thing needs to be set. Then USB debugging needs to be enabled in this device. You need a special USB driver to connect this. I guess this works if you build your own Android client. You could also run this in the simulator. If you're an existing APK, there's quite a lot of hacky-hacky to do this, but I just wanted to point to this. On iOS, iOS is also a little bit difficult. iOS has no real file system where you can access files, and if your iOS app isn't working, you cannot log into ownCloud when you can't place your files inside. Now, a little bit difficult. On iOS, there are two different kinds of files. One is a file that is produced when the app crashes. This happens not that often, but sometimes it uses closes or so. Those crash logs you can access on different locations. You can go to... If you attach the iOS device to an Xcode instance, you could select the device and get the crash log. Oh, there's a little error in copying and pasting. On your Android, on your iOS device, you could go to your settings, privacy, usage, submenu, submenu, submenu, and then you see all your crash logs, and you could get it and send by mail. The other thing a regular user could do is just sync the iOS device with iTunes, and then on the local disk of your computer and directory, there are also the crash logs. If you have your own iOS app in the App Store, and the users don't disable it, the crash logs get automatically transferred after some time to your accounts in iTunes Connect as a developer or as a distributor of an iOS app, you could get your information about the user. I'm not sure if we can trust the numbers. I guess on cloud users, most of them are sensible about your privacy. I guess a lot of them disable the hotel app and everything about me. So perhaps the numbers don't work here. Normally the iOS app don't crash, but there's something that isn't working perhaps with my own cloud instance. I don't know an easy way to get the activity in production transmitted from the iOS device to me. One thing that works pretty well is also attach the iOS device to your Xcode, with a USB cable, open the window in the devices menu, select the device, and then on the bottom, you need to expand the bottom section, and there your log is scrolling through, and at the end, at the bottom right, there's a button Save, and there you could generate a file that is live logging or the activity, and you could save and provide this information to engineering. So we could also get some information in between, between your own cloud, between the web server, between the clients. One tool I really love is Meet & Proxy. Meet & Proxy, MITM, it's man in the middle, so evil stuff, hacky tools. You run it on one of your computers, and for Meet & Proxy you have fake SSL certificates, and you have your device, serve a website for Meet & Proxy, and it asks you, it works also well on iOS, do you want to install this special certificate? And I say, oh, yes, nothing bad will happen. And then your device connects through your local computer with an on-cloud server, and you see every single HTTP, HTTP request as a line in Meet & Proxy, you could navigate through them, top and go down, and you can inspect every request and every response, but for us it's interesting, just click the W button, and this gets saved as a text file. So the text file is really ugly, for me it is hard to debug, but if you send this information to me, I will open my Meet & Proxy, I will load this text file with the uppercase L key, and I see the same view as you, with this information about other requests, and for every request, I click the Enter key, and then I see the request with all details, I see a response with all details, and I see all cookies, and this we used quite successfully in the last time to debug special Apache server configuration stuff. Yeah. Yeah, thanks Markus for the information. The last time Ms Reuser included this in the package, so this pieces this ugly text files directly, but if I wanna play, and look for myself, I prefer Meet & Proxy, but it's the same, so you can just... capture your traffic, save it, and this text file is identical, and load it in their own meeting box and have this bite good view about the traffic. And there you see, if it can't connect, if it gets redirected, does it get a cookie, doesn't it get a cookie or the information? So this is really, really cool. So the next thing, browser debugging. If Firefox, you have the developer tools, Firebug, Chrome, Safari, you have the web inspector. So it's pretty easy. You know, I use a record browser and then I see the network tab. And if I reload the page, I see all the requests. Here, every status is 200, which is nice. I should disable caching for this, but if something is not working, something is not loading, something missing, also if you play with a theming on cloud, then there you will see what is wrong. Or the front end guys, you could also inspect things. So you see here the document tree and on the right, you see all the CSS information about it and the commuted site, you see the size of the elements and so on and so on. But, for example, for the network view, what would be interesting for me, I don't know if they're good export. So sometimes I ask, yeah, couldn't send me a screenshot of your 404 results in this list. If some of you know a good way to export this and to investigate this, I would be very happy. So this is one of the question I don't know at the moment. Also the console, which shows JavaScript issues. Also included and very similar in, for Firefox. Okay. Another thing, Brody told us about and I don't know if there's another workshop for this. Very cool thing, framework for end-to-end testing, the core storage function of own cloud-based service installation, client sharing, automated tests, massive parallel testing your own cloud, when the grace condition ends on. Development from CERN, they are helping a lot to make own cloud better. On support perspective, it's difficult. So installing Smashbox itself, with the case, they had a lot of errors in the logs. But Piotre helped me in investigating and a lot of the errors were produced because of dependencies from Smashbox itself. So it had nothing to do with issues in the own cloud installation. And currently, yeah, this is not supported in the way as we support the own cloud instance. And yeah, if you test your Smashbox very extensively, you produce a lot of data. So I've got a link with 32 gigabytes of data and say, oh, look here, something is a burden, please investigate. So makes it difficult. But this is very interesting and we are checking if we include this, for example, of the own cloud deployment, could this instance be checked with an automated Smashbox test? Perhaps this is interesting. So yeah, we're still investigating it. Smashbox works best with Xdebug to really see what is going on there. Xdebug is a PHP module. It can be installed on all platforms, xdebug.org. And for example, if you combine it with PHP Storm, for PHP Storm, you could, there's a bookmarklet you could use and this sets a magic cookie and connects with your own cloud instance. It produces additional output and your local copy of PHP Storm get this, you know, the data of PHP Storm get this debug information from your remote on cloud instance with Xdebug enabled. You see all the variables set all the information from your own cloud. You could set break points. If you know, oh, the error is happening here. I could just pause it and it will stop here. I see the current state can debug this and so on. But this is all for nothing that can be easily forwarded to the own cloud support. So we are using it internally to investigate things. The interesting stuff just wanted to, if you're an outlook. Yeah. And that's it for now. If you have more information, we're here, just contact us. If you found something that we don't know, I would be very happy to learn this from you to help you better. If you have issues or experiences or something that's working at you, I'm very interested. Yeah. And at the end, this presentation is available on GitHub in my private account. Just download it, PHP as localhost and you can see it in your own browser. Yeah. Thanks. Thank you. Thank you.
 I come to my talk.  Getting some insight in all the parts or the components of the  platform from on cloud server and the client.  I'm Michael.  I work in the customer support team.  I also work with the branding service.  You will find him on GitHub.  Michael Stingel.  You will find me on Twitter.  Michael Stingel.  Elsewhere.  You can guess.  I sit down and collected all the stuff that could help to  find issues with your own cloud, with your clients.  So if something isn't working in your setup, it's not always  related to own cloud.  Sometimes it's a configuration issue in your web server or  something else.  I found quite a long list of places where you could get  information and for me as working customer support, also  sometimes we cannot reproduce the issue.  Users are reporting, also engineering.  Then we need your help providing us the information, send us  all the stuff that you can get so to make it possible to help you.  We work with engineering to find those issues and fix it if  necessary.  We will go through all of them.  What we have is our own cloud log.  If you run the own cloud server, there is a default in your  data director.  There is a bad example.  You shouldn't place your data director inside the HG Docs  folder.  But this is a default installation.  There inside there is a own cloud.log.  This is very easy.  On a basic technical level, you just go to the admin page, scroll  down and there you can see the latest log entries.  You can expand this view to see more of them.  You can just download the log file and you can configure the  logging level.  What should be included in the log file.  You have a pretty big own cloud instance and you enable high  level of logging.  It may slow down your own cloud a little bit but we have something  for you later.  Going a little bit deeper, also for the own cloud log, there  is something, some parts that we configure.  With the log type, log type is own cloud normally.  In the config.php file, we can define where should the log  file be placed.  Perhaps I need another location.  I have the log level, which is three at the moment, which is  default.  Level zero is everything.  With a log date format, you know, date, it's a little bit  different sometimes.  Some people would like to have their own cloud, not this file,  instead log type syslog, sending it to the syslog demon.  As I said, if you enable logging for every action on the  own cloud server, it may slow your instance down.  Again, there's a thing, not many people know, it's a log  condition.  You can place this in your config.php file and enabling  logging only for some users.  This user set here will become log level zero, so everything,  debug information.  Or if you want to investigate issues with a specific app, you  can enable logging for specific application.  For further debugging, you can add shared secrets to your  requests and to only investigate those requests and get the  full output in the log.  So, the next thing.  You need to add it in the HTTP request.  The question was about the shared secret.  It needs to be set in the HTTP request, so this is not that  easy to add this to your client if you don't build it yourself.  This is more complex on the technical side.  Using this for a specific user app is very easy.  Shared secret, this is more like debugging for development.  Does it answer your question?  More or less.  But limiting this to a specific user, it prevents all the  issues about slowing your instance down.  What other logs do we have?  We have the web server log, like example in Apache.  On the top, webdav is HTTP.  Everything we do, we will find the web server log.  For example, this is a little bit of the Apache log.  I see the IP address, user name of the client user.  I downloaded my hacker songs and seeing also I'm using the  latest version of the desktop client on my Mac.  Then we also have the time stamps and sometimes we have the  request triggering the action of the Apache server, but we  don't have the own cloud log.  We can compare the times and see something is happening there.  We have got all this information.  This is just like an outlook.  Analyzing the logs, like Horgan mentioned earlier this day,  there's for example the Elk stack, and now it's got the  Elastic stack.  This is a tool that could analyze huge amounts of log files.  You could define filters and visualizations to see how often,  on which time, from which user, from which location, from which  type of client, some incidents happen.  This is nothing we provide at the moment, but I would recommend  to check this out to see better what's going on in your own  cloud systems.  I know some of you are using this.  Let's go to the client side.  In the desktop client, there's an easy way to do this.  Just open the client setting, press F12 key, this window pops  up, and you can search in it.  You can clear again to remove all the previous stuff, and you  can save it as a text file, and then you have other information.  The problem is something happened.  Then I start logging, but I don't have the incident itself.  We have a few options to start the client with logging features.  The one is adding the parameter log file at the end, and defining  the location of the log file, and you can get, at the moment of  the start, a list of all the actions that happens in the client.  This could become a little bit bigger.  My favorite is the log tier option, where you define a directory,  and in this directory, you will get a bunch of files.  Every Synchron produces one file, and the next Synchron  produces a new file, which is, for me, easier to understand.  You can set the log expire in hours.  If you want to run this on a user's computer, you just say,  okay, keep it for 48 hours, and it don't blow your disk.  My favorite is defining the own Cloud Sync directory itself as  location for the log directory, so the client logs get transmitted  to me as an administrator of the instance, and then I could check  what's going on in this machine.  Yeah.  This comes with a desktop sync client on Android.  It's a little bit different.  It's pretty easy for the Android builds generated in ownPrander.  I showed you ownPrander before.  If you trigger a build in ownPrander, you get your APK,  and every ownPrander build has a twin, and in this debug.apk,  there's debugging enabled.  Just ask the user to run this APK instead, and later get this  log file from the Android device in the file system and let it  send to you or send it to ownCloud.  This is also possible to connect your device to Android Studio,  but that's a little bit more complicated.  The application itself needs to be debuggable.  There's a manifest file inside there.  This thing needs to be set.  Then USB debugging needs to be enabled in this device.  You need a special USB driver to connect this.  I guess this works if you build your own Android client.  You could also run this in the simulator.  If you're an existing APK, there's quite a lot of hacky-hacky to do this,  but I just wanted to point to this.  On iOS, iOS is also a little bit difficult.  iOS has no real file system where you can access files,  and if your iOS app isn't working, you cannot log into  ownCloud when you can't place your files inside.  Now, a little bit difficult.  On iOS, there are two different kinds of files.  One is a file that is produced when the app crashes.  This happens not that often, but sometimes it uses closes or so.  Those crash logs you can access on different locations.  You can go to...  If you attach the iOS device to an Xcode instance,  you could select the device and get the crash log.  Oh, there's a little error in copying and pasting.  On your Android, on your iOS device, you could go to your settings,  privacy, usage, submenu, submenu, submenu,  and then you see all your crash logs,  and you could get it and send by mail.  The other thing a regular user could do is just sync the iOS device with iTunes,  and then on the local disk of your computer and directory,  there are also the crash logs.  If you have your own iOS app in the App Store,  and the users don't disable it,  the crash logs get automatically transferred after some time  to your accounts in iTunes Connect as a developer or as a distributor of an iOS app,  you could get your information about the user.  I'm not sure if we can trust the numbers.  I guess on cloud users, most of them are sensible about your privacy.  I guess a lot of them disable the hotel app and everything about me.  So perhaps the numbers don't work here.  Normally the iOS app don't crash,  but there's something that isn't working perhaps with my own cloud instance.  I don't know an easy way to get the activity in production transmitted from the iOS device to me.  One thing that works pretty well is also attach the iOS device to your Xcode,  with a USB cable,  open the window in the devices menu, select the device,  and then on the bottom, you need to expand the bottom section,  and there your log is scrolling through,  and at the end, at the bottom right, there's a button Save,  and there you could generate a file that is live logging or the activity,  and you could save and provide this information to engineering.  So we could also get some information in between,  between your own cloud, between the web server, between the clients.  One tool I really love is Meet & Proxy.  Meet & Proxy, MITM, it's man in the middle, so evil stuff, hacky tools.  You run it on one of your computers,  and for Meet & Proxy you have fake SSL certificates,  and you have your device,  serve a website for Meet & Proxy,  and it asks you, it works also well on iOS,  do you want to install this special certificate?  And I say, oh, yes, nothing bad will happen.  And then your device connects through your local computer  with an on-cloud server,  and you see every single HTTP,  HTTP request as a line in Meet & Proxy,  you could navigate through them,  top and go down,  and you can inspect every request and every response,  but for us it's interesting, just click the W button,  and this gets saved as a text file.  So the text file is really ugly,  for me it is hard to debug,  but if you send this information to me,  I will open my Meet & Proxy,  I will load this text file with the uppercase L key,  and I see the same view as you,  with this information about other requests,  and for every request,  I click the Enter key,  and then I see the request with all details,  I see a response with all details,  and I see all cookies,  and this we used quite successfully in the last time  to debug special Apache server configuration stuff.  Yeah.  Yeah, thanks Markus for the information.  The last time Ms Reuser included this in the package,  so this pieces this ugly text files directly,  but if I wanna play,  and look for myself,  I prefer Meet & Proxy,  but it's the same,  so you can just...  capture your traffic, save it,  and this text file is identical,  and load it in their own meeting box  and have this bite good view about the traffic.  And there you see, if it can't connect,  if it gets redirected, does it get a cookie,  doesn't it get a cookie or the information?  So this is really, really cool.  So the next thing,  browser debugging.  If Firefox, you have the developer tools,  Firebug, Chrome, Safari, you have the web inspector.  So it's pretty easy.  You know, I use a record browser  and then I see the network tab.  And if I reload the page, I see all the requests.  Here, every status is 200, which is nice.  I should disable caching for this,  but if something is not working,  something is not loading, something missing,  also if you play with a theming on cloud,  then there you will see what is wrong.  Or the front end guys, you could also inspect things.  So you see here the document tree and on the right,  you see all the CSS information about it  and the commuted site, you see the size of the elements  and so on and so on.  But, for example, for the network view,  what would be interesting for me,  I don't know if they're good export.  So sometimes I ask, yeah,  couldn't send me a screenshot of your 404 results  in this list.  If some of you know a good way to export this  and to investigate this, I would be very happy.  So this is one of the question I don't know at the moment.  Also the console,  which shows JavaScript issues.  Also included and very similar in, for Firefox.  Okay.  Another thing,  Brody told us about and I don't know if there's another  workshop for this.  Very cool thing, framework for end-to-end testing,  the core storage function of own cloud-based service  installation, client sharing, automated tests,  massive parallel testing your own cloud,  when the grace condition ends on.  Development from CERN, they are helping a lot  to make own cloud better.  On support perspective, it's difficult.  So installing Smashbox itself, with the case,  they had a lot of errors in the logs.  But Piotre helped me in investigating  and a lot of the errors were produced  because of dependencies from Smashbox itself.  So it had nothing to do with issues  in the own cloud installation.  And currently, yeah, this is not supported  in the way as we support the own cloud instance.  And yeah, if you test your Smashbox very extensively,  you produce a lot of data.  So I've got a link with 32 gigabytes of data and say,  oh, look here, something is a burden,  please investigate.  So makes it difficult.  But this is very interesting and we are checking  if we include this, for example,  of the own cloud deployment,  could this instance be checked  with an automated Smashbox test?  Perhaps this is interesting.  So yeah, we're still investigating it.  Smashbox works best with Xdebug  to really see what is going on there.  Xdebug is a PHP module.  It can be installed on all platforms, xdebug.org.  And for example, if you combine it with PHP Storm,  for PHP Storm, you could,  there's a bookmarklet you could use  and this sets a magic cookie  and connects with your own cloud instance.  It produces additional output  and your local copy of PHP Storm  get this, you know,  the data of PHP Storm get this debug information  from your remote on cloud instance with Xdebug enabled.  You see all the variables set  all the information from your own cloud.  You could set break points.  If you know, oh, the error is happening here.  I could just pause it and it will stop here.  I see the current state can debug this and so on.  But this is all for nothing that can be easily forwarded  to the own cloud support.  So we are using it internally to investigate things.  The interesting stuff just wanted to,  if you're an outlook.  Yeah.  And that's it for now.  If you have more information,  we're here, just contact us.  If you found something that we don't know,  I would be very happy to learn this from you  to help you better.  If you have issues or experiences  or something that's working at you,  I'm very interested.  Yeah.  And at the end, this presentation is available on GitHub  in my private account.  Just download it, PHP as localhost  and you can see it in your own browser.  Yeah.  Thanks.  Thank you.  Thank you.
Michael_Stingl__Getting_Insights_from_your_ownCloud_and_Desktop_and_Mobile_Clients_NOPDf8zAqnQ_0_47.png Michael_Stingl__Getting_Insights_from_your_ownCloud_and_Desktop_and_Mobile_Clients_NOPDf8zAqnQ_48_107.png Michael_Stingl__Getting_Insights_from_your_ownCloud_and_Desktop_and_Mobile_Clients_NOPDf8zAqnQ_108_155.png Michael_Stingl__Getting_Insights_from_your_ownCloud_and_Desktop_and_Mobile_Clients_NOPDf8zAqnQ_156_203.png Michael_Stingl__Getting_Insights_from_your_ownCloud_and_Desktop_and_Mobile_Clients_NOPDf8zAqnQ_204_251.png Michael_Stingl__Getting_Insights_from_your_ownCloud_and_Desktop_and_Mobile_Clients_NOPDf8zAqnQ_252_299.png Michael_Stingl__Getting_Insights_from_your_ownCloud_and_Desktop_and_Mobile_Clients_NOPDf8zAqnQ_300_371.png Michael_Stingl__Getting_Insights_from_your_ownCloud_and_Desktop_and_Mobile_Clients_NOPDf8zAqnQ_372_575.png Michael_Stingl__Getting_Insights_from_your_ownCloud_and_Desktop_and_Mobile_Clients_NOPDf8zAqnQ_576_683.png Michael_Stingl__Getting_Insights_from_your_ownCloud_and_Desktop_and_Mobile_Clients_NOPDf8zAqnQ_684_935.png Michael_Stingl__Getting_Insights_from_your_ownCloud_and_Desktop_and_Mobile_Clients_NOPDf8zAqnQ_936_4991.png Michael_Stingl__Getting_Insights_from_your_ownCloud_and_Desktop_and_Mobile_Clients_NOPDf8zAqnQ_4992_6251.png Michael_Stingl__Getting_Insights_from_your_ownCloud_and_Desktop_and_Mobile_Clients_NOPDf8zAqnQ_6252_9203.png Michael_Stingl__Getting_Insights_from_your_ownCloud_and_Desktop_and_Mobile_Clients_NOPDf8zAqnQ_9204_10859.png Michael_Stingl__Getting_Insights_from_your_ownCloud_and_Desktop_and_Mobile_Clients_NOPDf8zAqnQ_10860_13163.png Michael_Stingl__Getting_Insights_from_your_ownCloud_and_Desktop_and_Mobile_Clients_NOPDf8zAqnQ_13164_15779.png Michael_Stingl__Getting_Insights_from_your_ownCloud_and_Desktop_and_Mobile_Clients_NOPDf8zAqnQ_15780_17303.png Michael_Stingl__Getting_Insights_from_your_ownCloud_and_Desktop_and_Mobile_Clients_NOPDf8zAqnQ_17304_18623.png Michael_Stingl__Getting_Insights_from_your_ownCloud_and_Desktop_and_Mobile_Clients_NOPDf8zAqnQ_18624_20267.png Michael_Stingl__Getting_Insights_from_your_ownCloud_and_Desktop_and_Mobile_Clients_NOPDf8zAqnQ_20268_20327.png Michael_Stingl__Getting_Insights_from_your_ownCloud_and_Desktop_and_Mobile_Clients_NOPDf8zAqnQ_20328_22727.png Michael_Stingl__Getting_Insights_from_your_ownCloud_and_Desktop_and_Mobile_Clients_NOPDf8zAqnQ_22728_26939.png Michael_Stingl__Getting_Insights_from_your_ownCloud_and_Desktop_and_Mobile_Clients_NOPDf8zAqnQ_26940_29027.png Michael_Stingl__Getting_Insights_from_your_ownCloud_and_Desktop_and_Mobile_Clients_NOPDf8zAqnQ_29028_29531.png Michael_Stingl__Getting_Insights_from_your_ownCloud_and_Desktop_and_Mobile_Clients_NOPDf8zAqnQ_29532_32195.png Michael_Stingl__Getting_Insights_from_your_ownCloud_and_Desktop_and_Mobile_Clients_NOPDf8zAqnQ_32196_32267.png Michael_Stingl__Getting_Insights_from_your_ownCloud_and_Desktop_and_Mobile_Clients_NOPDf8zAqnQ_32268_34847.png Michael_Stingl__Getting_Insights_from_your_ownCloud_and_Desktop_and_Mobile_Clients_NOPDf8zAqnQ_34848_35411.png Michael_Stingl__Getting_Insights_from_your_ownCloud_and_Desktop_and_Mobile_Clients_NOPDf8zAqnQ_35412_37043.png Michael_Stingl__Getting_Insights_from_your_ownCloud_and_Desktop_and_Mobile_Clients_NOPDf8zAqnQ_37044_37091.png Michael_Stingl__Getting_Insights_from_your_ownCloud_and_Desktop_and_Mobile_Clients_NOPDf8zAqnQ_37092_38063.png Michael_Stingl__Getting_Insights_from_your_ownCloud_and_Desktop_and_Mobile_Clients_NOPDf8zAqnQ_38064_38211.png
I come to my talk. Getting some insight in all the parts or the components of the platform from on cloud server and the client. I'm Michael. I work in the customer support team. I also work with the branding service. You will find him on GitHub. Michael Stingel. You will find me on Twitter. Michael Stingel. Elsewhere. You can guess. I sit down and collected all the stuff that could help to find issues with your own cloud, with your clients. So if something isn't working in your setup, it's not always related to own cloud. Sometimes it's a configuration issue in your web server or something else. I found quite a long list of places where you could get information and for me as working customer support, also sometimes we cannot reproduce the issue. Users are reporting, also engineering. Then we need your help providing us the information, send us all the stuff that you can get so to make it possible to help you. We work with engineering to find those issues and fix it if necessary. We will go through all of them. What we have is our own cloud log. If you run the own cloud server, there is a default in your data director. There is a bad example. You shouldn't place your data director inside the HG Docs folder. But this is a default installation. There inside there is a own cloud.log. This is very easy. On a basic technical level, you just go to the admin page, scroll down and there you can see the latest log entries. You can expand this view to see more of them. You can just download the log file and you can configure the logging level. What should be included in the log file. You have a pretty big own cloud instance and you enable high level of logging. It may slow down your own cloud a little bit but we have something for you later. Going a little bit deeper, also for the own cloud log, there is something, some parts that we configure. With the log type, log type is own cloud normally. In the config.php file, we can define where should the log file be placed. Perhaps I need another location. I have the log level, which is three at the moment, which is default. Level zero is everything. With a log date format, you know, date, it's a little bit different sometimes. Some people would like to have their own cloud, not this file, instead log type syslog, sending it to the syslog demon. As I said, if you enable logging for every action on the own cloud server, it may slow your instance down. Again, there's a thing, not many people know, it's a log condition. You can place this in your config.php file and enabling logging only for some users. This user set here will become log level zero, so everything, debug information. Or if you want to investigate issues with a specific app, you can enable logging for specific application. For further debugging, you can add shared secrets to your requests and to only investigate those requests and get the full output in the log. So, the next thing. You need to add it in the HTTP request. The question was about the shared secret. It needs to be set in the HTTP request, so this is not that easy to add this to your client if you don't build it yourself. This is more complex on the technical side. Using this for a specific user app is very easy. Shared secret, this is more like debugging for development. Does it answer your question? More or less. But limiting this to a specific user, it prevents all the issues about slowing your instance down. What other logs do we have? We have the web server log, like example in Apache. On the top, webdav is HTTP. Everything we do, we will find the web server log. For example, this is a little bit of the Apache log. I see the IP address, user name of the client user. I downloaded my hacker songs and seeing also I'm using the latest version of the desktop client on my Mac. Then we also have the time stamps and sometimes we have the request triggering the action of the Apache server, but we don't have the own cloud log. We can compare the times and see something is happening there. We have got all this information. This is just like an outlook. Analyzing the logs, like Horgan mentioned earlier this day, there's for example the Elk stack, and now it's got the Elastic stack. This is a tool that could analyze huge amounts of log files. You could define filters and visualizations to see how often, on which time, from which user, from which location, from which type of client, some incidents happen. This is nothing we provide at the moment, but I would recommend to check this out to see better what's going on in your own cloud systems. I know some of you are using this. Let's go to the client side. In the desktop client, there's an easy way to do this. Just open the client setting, press F12 key, this window pops up, and you can search in it. You can clear again to remove all the previous stuff, and you can save it as a text file, and then you have other information. The problem is something happened. Then I start logging, but I don't have the incident itself. We have a few options to start the client with logging features. The one is adding the parameter log file at the end, and defining the location of the log file, and you can get, at the moment of the start, a list of all the actions that happens in the client. This could become a little bit bigger. My favorite is the log tier option, where you define a directory, and in this directory, you will get a bunch of files. Every Synchron produces one file, and the next Synchron produces a new file, which is, for me, easier to understand. You can set the log expire in hours. If you want to run this on a user's computer, you just say, okay, keep it for 48 hours, and it don't blow your disk. My favorite is defining the own Cloud Sync directory itself as location for the log directory, so the client logs get transmitted to me as an administrator of the instance, and then I could check what's going on in this machine. Yeah. This comes with a desktop sync client on Android. It's a little bit different. It's pretty easy for the Android builds generated in ownPrander. I showed you ownPrander before. If you trigger a build in ownPrander, you get your APK, and every ownPrander build has a twin, and in this debug.apk, there's debugging enabled. Just ask the user to run this APK instead, and later get this log file from the Android device in the file system and let it send to you or send it to ownCloud. This is also possible to connect your device to Android Studio, but that's a little bit more complicated. The application itself needs to be debuggable. There's a manifest file inside there. This thing needs to be set. Then USB debugging needs to be enabled in this device. You need a special USB driver to connect this. I guess this works if you build your own Android client. You could also run this in the simulator. If you're an existing APK, there's quite a lot of hacky-hacky to do this, but I just wanted to point to this. On iOS, iOS is also a little bit difficult. iOS has no real file system where you can access files, and if your iOS app isn't working, you cannot log into ownCloud when you can't place your files inside. Now, a little bit difficult. On iOS, there are two different kinds of files. One is a file that is produced when the app crashes. This happens not that often, but sometimes it uses closes or so. Those crash logs you can access on different locations. You can go to... If you attach the iOS device to an Xcode instance, you could select the device and get the crash log. Oh, there's a little error in copying and pasting. On your Android, on your iOS device, you could go to your settings, privacy, usage, submenu, submenu, submenu, and then you see all your crash logs, and you could get it and send by mail. The other thing a regular user could do is just sync the iOS device with iTunes, and then on the local disk of your computer and directory, there are also the crash logs. If you have your own iOS app in the App Store, and the users don't disable it, the crash logs get automatically transferred after some time to your accounts in iTunes Connect as a developer or as a distributor of an iOS app, you could get your information about the user. I'm not sure if we can trust the numbers. I guess on cloud users, most of them are sensible about your privacy. I guess a lot of them disable the hotel app and everything about me. So perhaps the numbers don't work here. Normally the iOS app don't crash, but there's something that isn't working perhaps with my own cloud instance. I don't know an easy way to get the activity in production transmitted from the iOS device to me. One thing that works pretty well is also attach the iOS device to your Xcode, with a USB cable, open the window in the devices menu, select the device, and then on the bottom, you need to expand the bottom section, and there your log is scrolling through, and at the end, at the bottom right, there's a button Save, and there you could generate a file that is live logging or the activity, and you could save and provide this information to engineering. So we could also get some information in between, between your own cloud, between the web server, between the clients. One tool I really love is Meet & Proxy. Meet & Proxy, MITM, it's man in the middle, so evil stuff, hacky tools. You run it on one of your computers, and for Meet & Proxy you have fake SSL certificates, and you have your device, serve a website for Meet & Proxy, and it asks you, it works also well on iOS, do you want to install this special certificate? And I say, oh, yes, nothing bad will happen. And then your device connects through your local computer with an on-cloud server, and you see every single HTTP, HTTP request as a line in Meet & Proxy, you could navigate through them, top and go down, and you can inspect every request and every response, but for us it's interesting, just click the W button, and this gets saved as a text file. So the text file is really ugly, for me it is hard to debug, but if you send this information to me, I will open my Meet & Proxy, I will load this text file with the uppercase L key, and I see the same view as you, with this information about other requests, and for every request, I click the Enter key, and then I see the request with all details, I see a response with all details, and I see all cookies, and this we used quite successfully in the last time to debug special Apache server configuration stuff. Yeah. Yeah, thanks Markus for the information. The last time Ms Reuser included this in the package, so this pieces this ugly text files directly, but if I wanna play, and look for myself, I prefer Meet & Proxy, but it's the same, so you can just... capture your traffic, save it, and this text file is identical, and load it in their own meeting box and have this bite good view about the traffic. And there you see, if it can't connect, if it gets redirected, does it get a cookie, doesn't it get a cookie or the information? So this is really, really cool. So the next thing, browser debugging. If Firefox, you have the developer tools, Firebug, Chrome, Safari, you have the web inspector. So it's pretty easy. You know, I use a record browser and then I see the network tab. And if I reload the page, I see all the requests. Here, every status is 200, which is nice. I should disable caching for this, but if something is not working, something is not loading, something missing, also if you play with a theming on cloud, then there you will see what is wrong. Or the front end guys, you could also inspect things. So you see here the document tree and on the right, you see all the CSS information about it and the commuted site, you see the size of the elements and so on and so on. But, for example, for the network view, what would be interesting for me, I don't know if they're good export. So sometimes I ask, yeah, couldn't send me a screenshot of your 404 results in this list. If some of you know a good way to export this and to investigate this, I would be very happy. So this is one of the question I don't know at the moment. Also the console, which shows JavaScript issues. Also included and very similar in, for Firefox. Okay. Another thing, Brody told us about and I don't know if there's another workshop for this. Very cool thing, framework for end-to-end testing, the core storage function of own cloud-based service installation, client sharing, automated tests, massive parallel testing your own cloud, when the grace condition ends on. Development from CERN, they are helping a lot to make own cloud better. On support perspective, it's difficult. So installing Smashbox itself, with the case, they had a lot of errors in the logs. But Piotre helped me in investigating and a lot of the errors were produced because of dependencies from Smashbox itself. So it had nothing to do with issues in the own cloud installation. And currently, yeah, this is not supported in the way as we support the own cloud instance. And yeah, if you test your Smashbox very extensively, you produce a lot of data. So I've got a link with 32 gigabytes of data and say, oh, look here, something is a burden, please investigate. So makes it difficult. But this is very interesting and we are checking if we include this, for example, of the own cloud deployment, could this instance be checked with an automated Smashbox test? Perhaps this is interesting. So yeah, we're still investigating it. Smashbox works best with Xdebug to really see what is going on there. Xdebug is a PHP module. It can be installed on all platforms, xdebug.org. And for example, if you combine it with PHP Storm, for PHP Storm, you could, there's a bookmarklet you could use and this sets a magic cookie and connects with your own cloud instance. It produces additional output and your local copy of PHP Storm get this, you know, the data of PHP Storm get this debug information from your remote on cloud instance with Xdebug enabled. You see all the variables set all the information from your own cloud. You could set break points. If you know, oh, the error is happening here. I could just pause it and it will stop here. I see the current state can debug this and so on. But this is all for nothing that can be easily forwarded to the own cloud support. So we are using it internally to investigate things. The interesting stuff just wanted to, if you're an outlook. Yeah. And that's it for now. If you have more information, we're here, just contact us. If you found something that we don't know, I would be very happy to learn this from you to help you better. If you have issues or experiences or something that's working at you, I'm very interested. Yeah. And at the end, this presentation is available on GitHub in my private account. Just download it, PHP as localhost and you can see it in your own browser. Yeah. Thanks. Thank you. Thank you.
in your web server or something else. I found Or if you want to investigate issues with a client logs get transmitted to me as an this log file from the Android device in the file can access files, and if your iOS app isn't to... If you attach the iOS device to an Xcode something that isn't working perhaps with my own