Civic tech tools that did not meet expectations: Lessons learned from the field

1. Introduction

Civic tech tools are often not used to the extent that their creators hoped for. Based on a series of cases in Central and Eastern Europe (CEE), Balkans and Russia, this paper is examining why this is the case and provides some insights on how to adapt the tools to the arising challenges. The aim of this piece is to support learning and improvement in the civic tech community in the region and beyond.

The evidence from the field shows us that many civic tech tools fail. Sometimes this has to do with the technology used and the lack of appropriate tech skills to apply it, but most of the time, it boils down to the human element. Organisations are struggling with involving users into the design and implementation of products, and of maintaining a fruitful relationship with public officials, who are often the key data providers or the target of the project. Surveys show that only a minority of civic tech organisations do any user research before choosing a tech tool and even fewer test the tools with potential users. At the same time, the majority of them expect easier buy in from project’s stakeholders and users, than what happens in reality.

1.2 Methodology

This paper is based on desk research of academic and practitioner literature, and interviews carried out with organisations that develop civic tech tools. Dmitri Romanovski from TechSoup Europe, who commissioned this research piece, conducted preparatory interviews with stakeholders from civic tech organisations that served as a basis for the more in-depth interviews carried out by Alina Ostling. In total, six interviews with key informants from the civic tech community in CEE, Russia and Western Balkans took place, mostly by Skype. The sample was focused on civic tech tools, which have not lived up to the expectations of those who implement them. Considering the growing importance of the concept of adaptiveness and agile development in the field of civic tech, the author used parts of the framework developed by Prieto-Martin et al., within the Making All Voices Count programme, to analyse how organisations adapt to challenges.

2. Challenges

Several of the interviewees show a strong engagement and commitment to their cause, some of them to the point of using their free time and volunteering in order to make projects work. At the same time, they are frank and realistic about the various challenges faced, related to both the civic and tech-side. Several of them find it difficult to estimate how much financial and human resources will be needed to have a working tool at the stage when they seek donor funding. Unrealistic expectations about the tech skills of the end-users is also cited as a recurring issue. Most of the problems, however, are not of financial or technical but of human nature, such as limited interest from citizens, and lack of will of civil society partners and public officials to cooperate. The key challenges are summarised in bullet points and detailed in thematic sections below.

Tech skills

Technology is at the core of civic tech so it is not surprising that several of the interviewees have experienced tech-related issues. A key challenge that emerges from the interviews is the lack of technical knowledge among potential users or even within the organisation itself. Alexey Sidorenko from Teplitsa of Social Technologies (Russia) reports that the technology used for their website-builder Giger was too complex: “We squeezed in a bunch of technologies that were too advanced even for advanced end-users. As a result, only three (civil society) organizations took advantage of the services - all the rest abandoned.” Alexey attributes this issue to miscommunication with the external developers, but also thinks that developers have a general tendency to bump up tech solutions without considering the level of user skills. The next iteration, Kandinsky, built upon the lessons learned and allowed more than 50 organizations to set up the websites within months after launch.

Tip

Do user testing and research early on, also before launch, in order to identify problems that users might face and to adapt the tools to the level of user skills.

Engaging thematic NGO partners

In Bosnia and Herzegovina, the organisation Zasto ne? tried to localize the Alaveteli tool for freedom of information requests but could not find thematic partners that would take over and develop the tool. Today their platform that was called Pravo Da Znam (pravodaznam.ba) does not exist anymore.

“In the end, we didn’t do it: couldn’t’ raise funding on our own. There were also no other organizations from those who deals with this issue who would want to pick it up”, says Darko Brakn from Zasto ne?. The same happened with the tool for reporting problems in the Bosnian education system Popravi Mi Skolu: “(...) it was not our field of work. At the same time, none of the student councils or universities around the country didn’t want to pick it up. So it died without proper ownership.” In Darko’s experience, partnerships with other civil society organisations that are not well-defined usually fail.

Tip

Define or formalise your partnerships so all the partners are clear from the start about their role, responsibilities and contribution (in financial terms or in-kind) to the project.

Competing solutions

In Lithuania, the project Skaidrumo Linija (Transparency Line) was intended to alert about possible acts of corruption and find out how to deal with such violations. Transparency International Lithuania invested a lot of energy in developing the tool but received very few reports from users. According to Karolis Granickas, previously at TI Lithuania, the product was not needed for the ecosystem since there are other channels to report corruption.

In Ukraine, one of the leading tech NGOs faced competition from another organisation when developing a tool aimed to track, analyze, and visualize financial sources of Ukrainian political parties. “Our managements did not manage to join efforts or to divide the work across the two organisations. As a result, we competed for financial resources and it looked very strange in the media, when we presented our project, and then a month later, the other organization presented their equivalent.” The anonymous Ukrainian source recalls that it was a serious blow to discover that there are competitors in the civil society sector. “Worst of all, we failed to agree on further actions and to join forces” says the source.

Tips

Paulius Murauskas, from Transparency International Lithuania, running the Transparency Line, suggests to build on experience of colleagues from similar projects, look into thematic networks already in place, reach out and talk to people before and during the project.

The interviewed NGO from Ukraine is also of the same opinion. Despite their discouraging experience with competitors, they still think it is critical to interact with potential competitors, to negotiate, and to share experiences.

Under-usage

A very common problem is the limited usage of civic tech tools. “All developers expect their projects to go viral. But in fact it is not realistic expecting millions or hundreds of thousands users. For the type of tools that we are building it is reasonable to expect 1,000-5,000 visitors on the day of the launch”, says Alexey Sidorenko from Teplitsa of Social Technologies (Russia).

The Lithuanian Transparency Line received just a handful of reports from users. According to our interviewee, Karolis Granickas, “the incentive mechanism was not fully developed and citizens did not have a clear promise on what would happen next when they reported corruption”. The broken “feedback loop” between citizens and the government, meaning that citizens do not receive any feedback about their inputs, is reported also in other studies (Gigler and Bailur 2014). Closing the feedback loop is not solely the responsibility of governments; civil society and the local tech community should be aware that they play a critical role as “infomediaries” in communicating relevant information back to citizens (Gigler and Bailur 2014).

Citizens might also decide not to engage because of the local culture or political context. Our source in Ukraine reports that in recent years, a large number of civic tech tools have been created but most of them are under-used. After the Euromaidan revolution, the number of active citizens was expected to increase. However, with time, fewer and fewer people seem to be motivated to engage. “Even the most active ones lose their motivation because of the sick, old law enforcement and judicial system protect violators of laws. In such conditions, the struggle for justice rests on a small group of enthusiastic Romantics”, says the Ukrainian source.

Tips

Alexey Sidorenko from Teplitsa (Russia) recommends to do everything simply and user-friendly. Develop the tool together with users; and do not forget to examine if potential users have a cognitive barrier or other types of difficulties in using the tool.

Test the tools before launch with different types of sample target groups (10-20 persons) and with civic hackers, recommends Paulius Murauskas, from Transparency International (Lithuania).

Paulius also suggests to seek pro-bono help by reaching out to local groups. By his experience, volunteering involves and empowers people, creating a multiplier effect.

This section has shown that the civic tech community is working in a complex ecosystem with a lot of pitfalls. Next we introduce some lessons learned about adaptiveness that might help the civic tech community to counter difficulties.

3. Adaptation

In the civic tech field, there is a growing consensus that improved adaptiveness and learning should be encouraged within projects and within organisations. Civic tech initiatives are often struggling with complex social problems and this requires a particular ability to adapt and to learn from challenges by the persons running them. Above all, it requires being receptive to project users and context, a willingness to experiment, an ability to develop strong networks of support on the ground and to carry out advocacy campaigns.

The motto, quoted from a study by Prieto-Martin et al. is “Learn early, fail less”. The adaptation ought to start very early and continue throughout the project cycle. Starting out with developing smaller and simpler tools, and working iteratively and incrementally helps to increase the success rate and the quality of the outcomes. However, in practice this does not come easy. There are high barriers to adaptiveness in civic tech, such as short project timelines, lack of an enabling environment for learning, and rigid donor mechanisms for implementing and reporting on projects.

All of the interviewees agree that adaptiveness is important. Many of the civic tech tools are new applications for the (local) market so it is often difficult to predict how the public will react. This is why adaptiveness is crucial. Several of the civic tech organisations interviewed are also well aware of the concept of adaptiveness and agile development, and are using these methods in their everyday work. “The Scrum book was an eye opener”, says Alexey from Teplitsa (Russia), “Now responsiveness to user needs is our ground rule”. Several of the interviewed organisations are trying to adapt their projects already during the design phase, and then throughout the project cycle, for example by running user tests, and consulting with government and civil society stakeholders.

However, the interviewed civic tech organisations also face serious barriers when they try to adapt, including donor rigidness and limited sustainability. Most of the time, civic tech organisations can not afford to invest in extensive user or context research before they propose a tool to donors. Another point is that often it is not until the development of the tool has initiated, that the organisations realise exactly the amount and type of human resources needed, and how to involve users. According to Darko Brkan from Zasto ne? (Bosnia and Herzegovina), civic tech organizations have to learn how to predict the whole process, both before and after launch, and set out a long term strategy. “What comes after launch is more important than the development part”, underlines Darko.

“Specificity is the enemy of efficiency”

When Alexey Sidorenko from Teplitsa (Russia) said that “Specificity is the enemy of efficiency”, he meant that specificity of donor requirements is at odds with the efficiency of civic tech work. In line with the findings of Prieto-Martin et al., some interviewees emphasize that the donor funding is not always as flexible as needed. Alexey thinks that grants are generally not designed for agile development of tools. Larger donors have excessive control over projects, instead of focusing on goals. “The grant system is broken”, Alexey says, “performance indicators are too rigid and project proposals require too much detail”. According to Alexey, some grants are designed in a way that rewards the ability to do things on time rather than achieving impact. “The implementing organisations launch a tool, then have to move on quickly to another tool. If donors give additional funding, the tools can be improved, but this is not always the case”.

Tip

Darko Brkan from Zasto ne? (Bosnia and Herzegovina) recommends to build in resources for additional development time in the project plan and budget to have more flexibility, when challenges arise.

Convince donors to consider adaptiveness and negotiate funding not only for development but also for maintenance and PR, stresses Darko.

Sustainability

Several interviewees stress that the funding structure of civic tech tools often overlooks sustainability. Because of this, many tools do not make it much beyond the launch. “Most of the time it is unclear who will take care of the tools after development. Donors should evaluate the capacity of recipients to maintain the tools also after launch”, says Darko Brkan from Zasto ne? (Bosnia and Herzegovina). Meanwhile, civic tech organisations are also learning from past mistakes. The Bosnian NGO Zasto ne? nowadays always secures funding not only for development but also for maintenance and PR of the tools.

Tip

“Don't start working with new tools if you don't have sustainable and substantial funding”, says Vid Doria from Transparency International (Slovenia).

Alexey Sidorenko from Teplitsa (Russia) thinks that civic tech organisations should not rely solely on grants but examine broader options, including the commercialisation of the tool. For example by using the freemium approach with core optionalities for free and support against payment.

“Make tools ‘semi automatic’, so they require as little attention as possible in terms of staff hours”, so you can continue to maintain them, says Paulius Murauskas from Transparency International Lithuania