In the first episode of the StefTalks Podcast, we dive into the potential for agile methodologies to assist and expedite artificial intelligence projects and the challenges that may arise. Listen to our conversation with Fabio Caversan, the Digital Business and Innovation Vice President at Stefanini and Gercel Silva, Stefanini Agile DevOps Senior Solution Manager and Practice Lead here
Agile methodologies have long been praised for their ability to increase team collaboration, break down silos, empower decision-making and project management, and more.
With roots in product development, the agile methodology was later adopted in the nineties by software engineers who were experimenting with different approaches to software development. Today, you can find agile ways of working in many industries outside of IT. From agile marketing to agile HR, it seems that the agile methodology has something to offer everyone.
So, what happens when agile meets artificial intelligence (AI)?
Some argue that the agile methodology can accelerate AI projects through feedback loops that facilitate fast-paced problem solving. Others maintain that existing agile methodologies encounter significant challenges when faced with the unique lifecycle requirements of AI projects.
So, which is it? Can agile and artificial intelligence work together or are they completely mismatched?
To get to the bottom of this question, we recently tackled this topic on the inaugural episode of the StefTalks Podcast.
How can AI be used to improve your business? We have three ideas for you here!
Replacing Waterfall with Agile
Before companies began implementing agile, many organizations were slowed by traditional project management approaches known as “waterfall.” Modeled off assembly line methods of production, this approach trapped software projects in months – or even years – of the design and development process, testing, and deployment.
Agile, on the other hand, tight, is structured around short iterations that quickly generate a deliverable that meet the business’ needs. While process is important and continuously refined as the project moves forward, agile places an equally important focus on the interactions of team members, collaboration, and the ability to quickly respond to change.
Broadly speaking, the agile methodology is credited for its emphasis on flexibility. From Scrum to Kanban – both different frameworks in and of themselves – their strengths lie within their focus on continuous feedback, eliminating waste, and adjusting processes to deliver products to market more quickly and efficiently.
So, to summarize: if your business has a complex problem that requires intense cross-functional collaboration of experts, then agile can help you succeed.
Challenges Presented by Agile and Artificial Intelligence Projects
Artificial intelligence is nothing new; on the contrary, the technology has become increasingly integrated into our personal and professional lives. As companies focus more on developing AI, machine learning projects, and other cognitive projects, it only follows that new work processes need to be developed and refined to best support AI project management.
At first glance, it seems that agile is a perfect match for AI. After all, agility focuses on continuous improvement. It gives teams the opportunity to revisit their work processes, discuss challenges they encounter, and find new and creative ways of improving their efficiencies.
When it comes to artificial intelligence, however, more clarity is needed around certain AI initiatives to properly identify where agile can help – and where it might hinder the progress of some AI projects.
The challenges arise when we remember that agile was initially developed to support product development and software engineering. From specific team roles like product owners and Scrum to project management approaches that are application development-centric, some agile approaches are viewed as incompatible with the lifecycle needs of AI projects.
An Agile Approach to AI: It’s All About Data
Where did the clashes between AI and agile start?
Clearly, you can’t manage AI projects today with a one-size-fits-all, cookie cutter approach. Firstly, there’s the fact that many AI projects require you to develop new AI models, architectures or techniques. In these instances, what is being defined as a deliverable? How does a project that doesn’t produce a concrete result align with an agile framework?
Further, AI projects that require working in new machine learning models or performing heavy data analysis often are not compatible with the typical agile methodology. This is due to the variable nature of machine learning models and the time it might take for algorithms to produce results, which may not match the expectations set forth by the project.
In these cases, these projects require even more flexibility than the adaptability built into the agile model, necessitating a complete change in strategy.
To rectify this issue, the agile approach to project management needs to allow the data scientists that train the models to perform the full scale of their work. This includes cleaning and data preparation for a variety of data frameworks; having the time to regularly check the models and monitor how accurate their predictions are; and finally, with training data, refine the models to deliver the expected results.
Clearly, an agile approach to AI must consider the data science at the center of every AI project. By connecting data engineers and analysts with business goals, project initiatives become more transparent, tangible, and achievable.
Reach business goals with agile squads. Learn how here!
The Benefits of Agile and AI
What we’ve found is not necessarily that agile and AI cannot be compatible. Rather, agile methodologies need to be tweaked to accommodate AI projects.
First, the emphasis on communication is a huge advantage for AI projects. By helping teams work more intelligently and focus on value delivery, agile helps team members working on AI projects understand business goals and priorities more coherently.
Second, by keeping all aspects of the project visible and everyone involved, agility helps AI teams deal with unpredictable problems in a proactive way.
At the same time, automation powered by AI can be a huge boon to agile teams.
After all, agile teams need to have automated tests to expedite their progress. Typically, this role would be performed by QA (Quality Assurance), a systematic process that supports the project by looking at the team’s product, requirements, and goals. Then, the QA automates these tests by making sure that the product is running as expected.
We anticipate that advances in autonomous automation will expedite this process when AI can be plugged into software and create, if not all, a good part of that automation. After all, automating something is almost essential to agility. We’re talking about getting feedback and getting fast feedback.
When you have AI supporting this effort, AI can make that automation much shorter than if you had to have a QA programming all of the pieces of the test automation.
Meet SAI – Stefanini Artificial Intelligence. Learn more here!
