Intro: Broadcasting live from the Greater Perimeter. It’s time for Greater Perimeter Business Radio. Now here’s your host.

Lee Kantor: This episode of Greater Perimeter Business Radio is brought to you by Mirability. With their new compliance exo service taking you from it risk to it reward. Now here’s your host, Eric Bomans.

Erik Boemanns: So quality assurance, or QA is a key part of making sure your company’s products will work as expected and not damage your customer or your own reputation. It’s one of those things that everyone knows they need, but frequently skips when building a new product, especially in the software business. There are plenty of big examples in the news of the importance of QA. The CrowdStrike accident last year, for example, when companies realized they needed to be better at doing better, they need help and to provide that help. My guest today, Kelly Kierans and Scott Williford are both with CelticQA, a leading provider of quality assurance solutions and strategy. Kelly is president of CeltiQA and also highly involved in the Atlanta technology community. So, Kelly, how about a quick introduction of yourself?

Kelly Kierans: Well, I appreciate you having us here, Eric. It’s great to have this conversation, and I appreciate you bringing QA to the forefront of it, because unfortunately, a lot of times it’s an afterthought. But, uh, yes, I am Kelly Cairns. I’m the president of Celtic QA solutions. I oversee daily operations, sales and marketing and just basically work with the executive team to grow and scale the business. And we’ve hired on Scott Williford to help us in an advisory role. So, Scott, I don’t know if you want to introduce yourself.

Scott Williford: Yeah. I’m Scott Williford, uh, acting as CMO for CeltiQA, as well as their QA connector 2.0, which we’re launching and hopefully going to get a chance to talk about, but got decades of experience in the high tech startup space and the technology community and very well connected. Glad to be here.

Erik Boemanns: Yeah, well, thanks for both. Thanks to both of you for joining me today. And, um. Probably should just dive right in. So no, CelticQA has been providing QA. Solutions for the last 20 years. But can you give a little bit more background on the company itself?

Kelly Kierans: Sure. I can give you the story that is is not on the website and most people don’t hear or see. So, um, my husband and I actually own the business and, uh, gosh, forever we had talked about, what do we want to do? We wanted to start our own business, but we weren’t really sure, you know, what it was going to be. And, um, my husband at the time was working for Crawford Insurance Company, but he was working, um, for Keane. And they’re a big consulting company. And I at the time, I’ve actually done a huge transformation and shift in, in my life. Um, I at the time was actually working with orthopedic surgeons. Okay. Um, going in and putting neuromuscular stimulators on patients after surgery. So, um, we talked about it. We just, you know, sit at the dining room table. What do we want to be when we grow up kind of thing. And we talked about owning an Irish pub at some point because his uncle owns several Irish pubs up in New York, and my husband was born and raised in Ireland. So that was something we considered, but decided we didn’t want that lifestyle.

Kelly Kierans: We both played soccer and we thought, well, maybe we’ll open an indoor soccer complex someday. And that’s still on the roadmap. We might do that when we retire, but, uh, we’re really, um, the opportunity or we saw a gap in the marketplace, um, because of my husband’s work with, um, QA being kind of an afterthought. And he kept, you know, he’d come home and we’d have conversations and he’d complain about it or, you know, talk about it. And me being that I was in sales at the time, I thought, well, why don’t we do that? Let’s see if that’s something we can build. And, uh, so he, you know, he’s talked to the CIO he was working with and said, if if I do this, you know. Would you work with me? And the CIA was like, absolutely. But you have to leave the company that you’re with because there’s, you know, non-competes and all that sort of stuff. So on a handshake and a prayer and lots of faith, we took that leap and started our company back in 2005.

Erik Boemanns: That’s pretty impressive. Yeah. And I think it’s great that you mentioned the idea of of the gap in the market. Right. Because there really is one when it comes to quality assurance. The challenge then is, of course convincing people that they need to add that quality assurance. So what are some stories that you tell your your potential clients or your clients to help them understand the importance of quality assurance?

Kelly Kierans: Well, I think it’s more if they have the the mindset, like a lot of times they’re experiencing the pain, but maybe not necessarily knowing how to fix that pain, and they’re just throwing people at it or tools at it or, you know, different things like that. So when we have the conversation with our clients, it’s more about what’s your QA strategy, you know, do you have an overall roadmap for QA? Are you putting a focus on that, or is that just something that you’re doing at the end and just trying to bring that conversation to the forefront and making them realize that that’s something that’s super important for their projects? Because honestly, if if they shift the focus from dev to QA and have a QA mindset first, if they begin with the end in mind, then their projects can and should go live with zero critical or major defects, right?

Erik Boemanns: I do like that you also mentioned the quality assurance strategy, because I think when I think about quality assurance companies, I do think about throwing people at right there. They have contractors, they have staff. They’re staffing companies basically.

Kelly Kierans: Right.

Erik Boemanns: And I know Celtic QA is not that. So maybe a little bit on that.

Kelly Kierans: Yeah we’re different from that. So and actually that’s one of the the big things that we, we, we’ve, we’re kind of disrupting the marketplace in a sense from that standpoint because the typical mindset is just hire someone, um, or an offshore company or something like that. So we come in and we’re very streamlined and efficient. We don’t bring in a whole huge team. We don’t bring a school bus of, you know, people in to work on the projects. Right. So we come in and we we work directly with the CIO or IT leader, you know, whomever is in that position and really help them figure out what their goals are, what they want to achieve, and then work with their teams in order to figure out, okay, what what are we missing? What’s the skills that are missing? What tools? Maybe they were sold that aren’t necessarily the right tools for their environment. So we’ll come in and assess that. But we do that as we’re working on a project. So we roll up our sleeves, lead, manage and execute entire projects. But then we’re able to see where the inefficiencies are so that the next project can go better, faster and more efficient.

Erik Boemanns: That completely makes sense. And so earlier you mentioned a little bit about what Paying the customers. Your customers are feeling. Your clients are feeling. And so obviously, if your product breaks the production system of major companies, that’s an obvious pain that you’re now feeling from a quality perspective. But what are other examples, maybe less noticeable examples of pain that companies may be feeling? That is a great reason they should be reaching out to you.

Kelly Kierans: Well, I think that’s a good segue and transition into QA connector, to be honest with you, because one of the other gaps that we saw, you know, in the 20 years of working on client projects were the tools were not designed for QA people. Yeah, they were designed for developers. So with seeing that gap and seeing the inefficiencies and seeing the issues that that caused, again, we were like, why don’t we solve that problem? So we started developing QA connector. And honestly, for years we’ve been using it internally ourselves on client projects. Um, but we didn’t have a commercial grade version. So we we’ve recently developed a commercial grade version of it to help from, from the full, um, lifecycle of QA. So it starts from the requirements all the way to execution. Um, and we’ve incorporated an AI piece which we can get into that a little bit more. But I think that was a big pain. Right. And and solving that with a test management tool that’s built by QA people for QA people we think is the solution.

Erik Boemanns: Yeah. And I know I actually have a software development background myself. And so yeah, exactly what you’re saying. The project management almost always is focused on developers. And then can can the QA team also use that, or can they use JIRA in the developer mindset to create issues? Can they use something like an Azure DevOps? Whatever they’re using for story tracking becomes the bug tracking as well, right? And and since its focus was on the development team, it’s an afterthought, a secondary capability that those products have. It’s not focused on QA. So I definitely think I see that right.

Kelly Kierans: And the nice thing about having a test management tool that’s focused on QA is it gives you a full audit trail. So, you know, when auditors do come in, especially if there’s, you know, compliance and, you know, all those Sox compliance, those types of things, you need to have that traceability and trackability. And a spreadsheet is really hard to do that with. Right. And those other tools, it’s just not designed to shut down those audits where our QA connector tool can shut them down quickly.

Scott Williford: I actually think, um, just going back to the question about other issues is like usually or often, and this has happened in my own career, where the developers had a small group of guys that did the QA, but it was still the developers doing QA, and that’s kind of indicative of the way the tools have been set up. Um, and really, you can’t test your own code because you’re only going to look for the thing you fixed, not the things that affected, that were affected that you fixed. So you know which by bringing that into an external resource or bringing that into a separate management tool. As as Kelly said, it gives you more coverage, it gives you traceability, and it really allows you to design and build test cases around every single, uh, requirement, not just those you’re touching. And so when you do your regression testing, you don’t forget to do the regression testing. It’s really a it certainly saves a lot of time and money and you know, oh, you fixed this. But now this isn’t working. It kind of gets gets fixed and found because you’re doing it properly and using best practices for QA.

Erik Boemanns: Right. And take a step back to just write for the moment. When we talk about quality assurance and software, it can be a little bit more abstract, right? Quality assurance, quality control and manufacturing. Right. There’s a part and it has to fit certain criteria. And if it doesn’t, you reject it. If it does, you accept it. Right. That’s pretty easy to understand. If we think about the software side We talked about, um, different coverages. We talked about the idea of the developer can’t test their own code. Great example of that. If you’re writing your own code, you probably only think about when it works. You don’t think about, well, how can I break it? How can I negative test. Exactly. And so when you’re in the software world, having a person who’s focused on quality assurance helps you think of all the other things, all the other things that people are going to be doing with the software that that developer never thought of. And so I think it’s important to when we say coverage, right? We’re talking about thinking about how is the software going to be used correctly. Does that work and how is it going to be used incorrectly. And does that also work the way it should work? And so I’m curious your thoughts on both the products QA that you’re building and also from your strategy, how do you think about coverage and quality assurance from the software perspective?

Scott Williford: Well, the the cool thing about QA connector in our I, we call it test gen. I built into it. We’ve used 20 years of experience to write the prompts. Okay. So, you know, you could go to ChatGPT and say, give me some test cases. Uh, but if you didn’t have that 20 years of experience, then you’re not necessarily going to get the best test cases. So what we’ve done is we’ve taken our 20 years of experience in testing software to make sure that when we’re talking to the AI, we’re giving IT requirements and saying, give us every negative test, test case, give us positive test cases, give us those abstract test cases. And, uh, we were we were running a test just yesterday where we literally took, um, some, some very simple basic requirements, you know, a couple of page word document uploaded it, and it came up with 85 test cases. But if we just sat there with a whiteboard, we may have come up with like 30. Yeah. And, and then we would have had to go to every semi to get figure out what test data do we need and all of that. And the AI did all of this in about six minutes, including what data you need. Um. It really, it really blows my mind is how much faster it is. And if since you’ve been in the software space, you know that when you’re really doing robust test cases, you got to go back to all the subject matter experts, all the SMEs, and it might take days, if not weeks, to get a final set of test cases to give QA with using test gen, I built into the QA connector. Literally. It took minutes. And then we’re saying SMEs approve these, right?

Kelly Kierans: So it solves that pain. It solves that problem.

Scott Williford: And it saves. It’s you know, it’s kind of one of those catch 22 because Celtic QA also provides services. And they would charge for all of that. And now they’re able to do it in six minutes where it might take, you know, a couple of days. Uh, but it’s it’s about bringing value to the client and it’s about bringing value. And we know that there are companies that are not going to use Celtic QA for testing, but they want to use QA connector 2.0. We had a really good meeting last week with somebody that that’s exactly it. I mean, they they have their own test team. They have all that. They just want the software.

Erik Boemanns: So let’s dive in. You brought up a great point about the the services would decrease. But when I hear we went from 30 test cases that we might have thought of organically to 85. Am I increasing the workload on my QA team or am I actually like I’m sure I’m improving the quality of the product, but now is the QA process slower because there’s all these additional test cases to go through?

Kelly Kierans: That’s where automation comes into play. So the test management tool that we have developed actually integrates in with automation tool. So that speeds up that process right.

Erik Boemanns: Yeah. So it’s really increasing the quality but not increasing. It’s not slowing down the QA process.

Scott Williford: No no. If anything it’s speeding it up right. Because now you’ve got you’ve got you you’ve got a higher higher quality and better coverage. So you’re not going to do as many cycles to find all the bugs. Does that make sense. Whereas before you might run into a bug like with a the 30 test case scenario, you might run into a bug, you know, two weeks after release or in the next version that you didn’t get coverage on the first time. So in theory, you’re testing fewer times, but you’re testing a deeper, a deeper or wider and a deeper set of test cases.

Kelly Kierans: Yep. The other thing is we have what we call a test bank. So what’s missing and what we’ve seen, you know, with others that have failed, other projects that are tools that have failed is you lose that historical information. Right? What you’ve what you’ve put together from past projects, that sort of thing. It’s either extremely hard to access, especially if you’re using Excel spreadsheets or you’re in JIRA or Ado. So we have a test bank that captures and holds all of that information. So if you want to go back from a sprint or another project and utilize that, it’s just as simple as clicking a box to pull that over.

Erik Boemanns: Gotcha. Um, so one of the things I’m thinking about as I, as as we talk is the CIO or the product leadership, whoever is in charge of software development at the organization? Uh, you mentioned earlier, you might find that issue two weeks later. So it becomes very reactive, right? And so I feel like this gives us an opportunity to become proactive maybe.

Scott Williford: Yeah, exactly. And one of the things that’s really, um, one of our first demos early when the product was demo able. Right. Uh, was with a well-known CIO here in Atlanta, and they said, I love the dashboard because now I can be the good guy in the QA meeting being part of the solution, rather than pointing out all the problems that is happening in QA. Because before, uh, using a tool like QA connector, you didn’t really the only insight you had was asking negative questions. And now there’s this unfiltered real time dashboard on every test and every tester, and it goes down to the requirement level. Project level. Overall level. And and you’re able to just see the trends see the tendencies. And so now management looking at it can be very proactive rather than reactive about hey it looks like we’re going to miss this deadline. What do we need to do to make it. Or it looks like there’s a set of common errors or common defects happening in this one section. We need to put more developers on that to to solve these problems so that those that set of defects is cleared up quicker.

Erik Boemanns: Got it.

Kelly Kierans: Yeah. And it’s unfiltered information. So you’re seeing it in real time so no one can hide anything and say, oh, the project’s doing great, when really it’s not. So this gives everyone the same visibility.

Erik Boemanns: Yeah. Well, and you just said a key word there that if we go back to your comments about auditing and reporting, you said unfiltered just now, but maybe what if I do want that filtered report? What if I am presenting to the auditors? I probably do want to have a bit more structured. How do you approach the reporting side of that? You touched on it earlier, but curious.

Scott Williford: So, uh, one of the things that the system does innately is that it manages the test plans. And then those tests as they’re being executed, it’s managing the execution. So, you know, who touched what, when, where, and it’s all stored in the data. So the dashboard that we’ve created gives you a certain level of insight for management. But when you’re looking at an audit trail there’s detailed data on this person. Tested it on this day. This was the result. This person tested the same test on this day. This was the result. All the date trails defect was created sent back over to Ada or to JIRA. Developer fixed it, came back to QA connector QA connectors. Now selling the tester to test it again. Better results. And you see all of that now, how you filter that is you could you could take it as it is. Or if you want to say, hey, we went through all these things and here, you know, so you do have some flexibility from exporting and then sorting the data if you will. But the nice thing is the auditors as deep as they want to look, we know when, where, what, how much time was spent. There’s no way to hide that because the tester is using our tool as he’s executing the tests.

Erik Boemanns: Gotcha. So we’re going all the way from executive dashboards all the way down to the exact granular detail.

Scott Williford: Absolutely, absolutely. I mean, it literally, this is this is something we were we were showing it yesterday. Uh, he was like, so this theoretically could help me know how to staff my testing because I can see how long these tests are taking. And so I can adjust my staff accordingly or reassign test cases or test plans to different testers, because these take a little bit longer because it it not only date stamps, it it times how long it took you to do the test.

Erik Boemanns: And it also helps them make that decision between manual versus automated. I assume at that point.

Scott Williford: Yes, exactly, exactly. And it gives you the ability, even in the dashboard to know this. Many tests were covered by automation, by scripting versus human interaction. But a human.

Kelly Kierans: You know, still utilize those tools. But there’s that integration between the two. So QA can use a tool designed for them while the developers are still utilizing their tools. And, and they speak to each other and pull the information back and forth. So it’s it’s a perfect marriage.

Scott Williford: Yeah. And we map the fields appropriately. So requirements stories and epics are translated to requirements. Then requirements generate test cases. Right. And so you could let’s say you’re smees and your developers are building out all the requirements in JIRA or Ada. It doesn’t matter. You could then connect a QA connector project to the JIRA or Ada project. Object. Import all of that data, run test gen AI. It’ll generate all the test cases and QA connector. And then when a defect is identified, it goes back. When the defect is updated, it’s updated on, you know, it’s synced back. One of the questions that comes up often right now, the way we’ve done it, it’s not a scheduled update, a scheduled sync. And the reason we chose that it was an intentional decision is that we want to be able to have the ability, let’s say, for example, in Ada, tickets are created by developers, so they’re not always put in the right place. So we don’t want to sync those and then have to resync those, because then it’s just the data is going to get convoluted. And so this way it gives you some control over hey it’s time to sync up and the like, you know, the same thing happens with defects going the other way. You want to have somebody who’s responsible to review those defects, to say, oh, these are really the same defects. Let’s convert. Let’s combine those, um, even though you, you’re supposed to link them, but humans are involved, so it’s not always done the right way.

Erik Boemanns: Yeah, absolutely. So connector new service from Celtica. Right. And um, if anybody is interested in learning more, how do they find out about. Where do they reach out to.

Kelly Kierans: They can reach out to me at Kelly Kierans at Celtica. Com. Or they can find me on LinkedIn. Um, that they can just find me and reach out to me.

Erik Boemanns: Yep. And just in case somebody misunderstanding you mind spelling the the name of the company, too?

Kelly Kierans: Oh, absolutely. Yes. Some people say Celtic as well. We say Celtic, but it’s c e l t I c q a.

Erik Boemanns: Okay. And that will have information on Scx connector and the other services.

Kelly Kierans: It’ll have information on the services. We’re building a separate website for Scx connector, but they can still contact me through the website.

Erik Boemanns: Awesome. Awesome. Um, so I want to go back to your story. Kelly. As a founder, a lot of the guests that I have here are founders, and you’ve been doing this a lot longer than many of them. Um, so I would love to hear some guidance that you would give people, especially if they’re just starting out a startup. It’s probably a software company, either from your own perspective as a founder or even from a quality perspective, because that’s going to be something that’s competing with all the other, you know, small business issues as you start that company is quality, taking a seat on the back burner. So just curious your thoughts.

Kelly Kierans: Well, there are a lot of startups that are developing software, right?

Scott Williford: So I’ve been in several.

Kelly Kierans: Um, excuse me. They they should absolutely think of quality first. Right? So they’re so excited about getting it developed and out there and that sort of thing. But you have to think about your brand reputation as well. And you want to put something out there that is quality because the way people are nowadays, if it doesn’t work, they’re going to move on. Um, if they have an issue, they’re going to move on. So you really do want to put that the quality first. So of course we would love to be able to work with everybody. Um, but the smaller companies, if they’re looking for a tool, they can use the QA connector tool. If they’re looking for services to help them have the best practices, you know, put, you know, a company that will scale and grow as they scale and grow. Um, we’d certainly love to help them out with that.

Scott Williford: I think also, having had some experience doing technology startups, we were in Atdc, a atdc company, back in the 90s, and then I did another startup, uh, all software based, um, one went from two engineers and me to 4.5 million users in three years. So we blew up and one of our advisors said to us is that because the developers were like they were worried they’d finish and then what would they do next? And one of our advisors told them said, look, development is never done. There’ll always be new features and there will always be new bugs. The only way to really in development is just to kill all the developers, right?

Erik Boemanns: Cancel the project.

Scott Williford: Cancel the project? Exactly. And in having been through that, what we what we realize and again, both of those startups were were they were successful and I had successful exits with both of them. Um, the, the the thing that we tried to focus on is making sure you get to that minimum viable product with as few bugs as possible so that you could sell it. But then when you released it, you wanted to make sure there was even lesser, even less bugs, because a lot of times you don’t recognize a bug as a bug. It’s just a feature request, right? And so you’ve got to be able to gauge that. Um, but I think a lot of times you get analysis paralysis, uh, as a startup by you’re trying to make it perfect and, and you want to get that’s where that minimum viable product comes in. Uh, getting a product. We’re running into that with QA connector right now. We keep we’re like.

Kelly Kierans: Oh, that.

Scott Williford: Would be cool. We’re like.

Kelly Kierans: We’re.

Scott Williford: Like, we got to get this ready for release.

Kelly Kierans: We’re like, no, we need to sell it.

Scott Williford: Yeah, we got to get this ready for release. And we were testing our own software with our software, which is kind of funny, but but we’re, uh, you know, we’re trying to get ready for release. And then we have some really big fish that want us to integrate with another project management tool. And so we’re, we’re we’ve decided we’ve drawn a line in the sand and said, uh, we are production ready. Uh, we’re going into full beta right now. Uh, the product is ready to be released to the production environment. So we’re going to have customer beta, uh, and, and we integrated with Jira and we integrated with Ada because those are the two most popular. But we’re going to come back and open up that API so that we can integrate with other tools. Uh, on the scripting side, right now we’re going to integrate initially with Ranorex. Um, but then because that’s the tool that we use internally, uh, but we really want to become the hub of all QA A software, and that means we’re going to have to touch various levels of project management tools, as well as automation tools and AI tools. Um, but it’s it’s been a fun project. It’s been a lot of fun. We’re we’re right there. We’re we’re bringing customers on right now.

Erik Boemanns: That’s that’s great. And a quick clarification, you did say happy to work with those small midsize businesses. But this is also enterprise ready as well I assume.

Kelly Kierans: Oh 100%. Yeah yeah yeah for sure. That’s who we’re talking to right now are the large enterprise businesses.

Scott Williford: Yeah we have we have several uh, in our beta pool. I there are they’re they’re they’re recognizable name brand global brands. Um, and they’re going to use it on a project, but they’ll go for us and, and you didn’t ask but we’ve, we’ve kind of looked at the marketplace and we’ve priced this in a way that you could start with five users and you can go up to enterprise level. How many users do you need?

Erik Boemanns: Gotcha.

Scott Williford: And it’s affordable. It’ll it’ll pay for itself just in the time you save by generating test cases using our AI tool.

Erik Boemanns: Right. And then reduced problems as soon down the road, too.

Scott Williford: Exactly. Exactly.

Kelly Kierans: Absolutely. Yep. Yep.

Erik Boemanns: So I was going to ask if there was anything else that you did want to share. So I think that was one thing. Anything else from you, Kelly?

Kelly Kierans: Well, I think, um, really, the the goal is simple, right? So we want to deliver high quality applications with zero critical or major defects. Um, we want to incorporate best practices. Uh, we want to have a QA tool for QA people. Right. And, um, and, and just bring about a smarter way to manage quality.

Erik Boemanns: That makes sense. Well, thank you both. I’m sorry. Thank you. Yeah. Thank you for being here today.

Kelly Kierans: We appreciate your time.