QA Blog Posts

January 13, 2020

Human Testing better than Automation

Recently Ministry of Testing asked:

From a testing perspective, what are humans really good at?

Based on many years of testing, here are my four things that humans testing is good at. Automation is a great tool, but for these items, they are no match.

Human Testing Approved

Four Ways Human Testing is Better than Automation

Exploratory Testing - Looking for ways to break functionality is best done with human testing. People don’t always use the conventional path when using a website. Exploratory testing by humans can find unique bugs.

Debugging the route cause of a bug. Humans can use all sorts of methods to discover why a bug might happen. Humans can use the Chrome Console, log files and visual logic to better understand the root cause of a bug.

New Product/Feature Testing - It’s better to perform manual testing when a product/feature is new. The feature could go through numerous changes - so human testing would best to start before investing in automation time.

Third Party Tools Integration - Using third-party tools that require logins. There’s a chance that third-party companies could make changes that will break the automation flow. (Such as changing ids or layouts) Human testing can help bypass any complexity that third party websites have.

January 6, 2020

Apple Numbers (QA Fail)

Apple Fail2020

Apple makes things easy - which is why I like using it as my computer platform. Just about everything is easier to do on Macintosh.

However, they seriously failed with making graphs in Numbers.

On Christmas Day, I was trying to create a chart of how early my daughter would wake up Christmas morning.

I was struggling with generating a chart from a bunch of data. Basically it was a spreadsheet that had years as one row and times in the second row.

Here's a sample shot of the data:

Christmas Numbers

When I clicked on the "Insert Chart", this is what I got:

Apple Number Chart

Looks like I'll have to do a lot of chart manipulation to make this to work. (I tried putting the data via column view and got the same results.)

Google Sheets

I put in the exact same data in a Google SpreadSheet, select the fields and clicked "Insert Chart" and got a perfectly matched chart:

Google Sheet Christmas

What I Learned

Apparently if I need to create a quick chart, the way to go is to use Google Spreadsheet (or Excel). Apple Numbers isn't all that user-friendly when it comes to creating charts.

Apple should make it simple to create charts - they do have some unique layouts/styles that aren't available in other applications. I shouldn't have to be a chart master to make it work - especially if the data is simple.

December 30, 2019

The Best 2019 QA Posts

This year I posted a lot of useful information about QA software testing. Along the way, I learned a lot more about new techniques and testing strategies.

Q A Approves

2020 Goals

Very excited about some of the things I learned this year. I'll keep the QA Monday topic going until the end of February.

Three Specific QA Goals for the next set of posts:

  • Discover some of the Basics of QA Testing.
  • How can testers become better testers.
  • What tools are available that people may not know exists.

I'll focus more time on the quality of content, not just to post because I have to post.

December 23, 2019

Add Search to Chrome

Did you know that you can search through many websites without actually navigating to them? This cool trick is beneficial if you wanted, for example, to go directly to the Wikipedia article on orangutans without visiting Google.com or Wikipedia's front page.

Setting up the Search

First, go to Settings > Search engine > Manage search engines. There, you'll see your default search engine (the one Chrome uses whenever you type a query into the search), other sites already available for quick searches, as well as the option to add other websites to the list.

To prep up for this, go to a site that you want a "quick search" and search for something using the search engine. Copy the URL that appears on the first page after you click on submit.

  • Click Add
  • Enter in a search engine name
  • Enter a KeyWords - Usually just the search engine name
  • Then enter the URL and replace the old search term with %s

Add Search Engine Chrome
Sample Search Entry

So, for quick Wikipedia searches, click "Add" under Manage search engines and add the site. Then, the next time you type "Wikipedia.org" in the search text, on the far right, you'll see a prompt telling you to press tab to search within the site. Once you press the tab button, you'll see "search Wikipedia" written in blue on the left side of the search field; type your query and Chrome will only search within Wikipedia.

This function isn't specific to reference or search sites. You can use the search field to search directly through nearly any site"even PCMag.com"as long as it's included in your managed list of search engines. Chrome will automatically add these "keyword searches" to any site you visit.

To do this manually, add "site:" to a query. Say, for example, you want to see everything PCMag has written about fitness trackers, you could type "fitness trackers site:pcmag.com" into the search field, and Google will return results from their site.

December 16, 2019

Incognito in Chrome

Google's Incognito Mode is a quick way to go "private" in Google Chrome. It's a way to browse the internet without keeping any history on your computer.

Google Description

Simple description of Google's Incognito from Google:

If you don’t want Google Chrome to remember your activity, you can browse the web privately in Incognito mode.

Incognito

Five Things I have learned while using Incognito

  1. You can only have one active Incognito session open at once. Which means that you can't open multiple Incognito windows with different Facebook logins.
  2. To Launch the Incognito mode simply Press ⌘ + Shift + n.
  3. Did you know that Websites knows that you're in Incognito mode! This may impact you're experienced with the websites. For example, if you try to create a new Facebook account, Facebook will make your login experience a bit more adventurous.
  4. Google Extensions like Go Incognito let's you open the current page in Incognito mode.
  5. Some websites will set prices when browsing normal vs Incognito mode. Always good to see what the price of something is when your not logged in.

What happens when you browse privately

Chrome won't save your browsing history, cookies and site data, or information entered in forms. Files you download and bookmarks you create will be kept.

Your activity isn’t hidden from websites you visit, your employer or school, or your internet service provider.

December 9, 2019

Misconceptions that exist within software testing

Someone recently asked, "What are some of the misconceptions that exist within Quality Assurance Testing?"

Some QA misconceptions that I have seen/encountered over the years:

QA Can Be Done By Customers

Let's save money and time. Let's have the customers test this feature.

Big mistake to have paying customers test untestable code is very risky. They may encounter some bugs that may block them from performing crucial tasks.

In addition, you may introduce code into production that may not be reversible.

Let's Tell You How Long it Will Take

Some Developers and Product team members may say, " The feature shouldn’t take long to test, let the lead developers figure out how much time to test a feature. "

In reality, The only people that know how long to test something are the people doing the manual testing.

QA is Easy, Anyone can Do it

Some people think, Anyone can QA, that's an easy job.

Nice thinking, it's almost saying anyone can paint a wall. Sure anyone can. However, those that know what they are doing, actually do a better job and will produce a better quality product.

Don't trust your work to anyone, use someone that knows the technology and can better judge the work.

December 2, 2019

GodFather Quotes

Here are some quotes from the movie GodFather but changed for QA. After each quote is a brief description of the logic.

The GodFather is a crime novel by Mario Puzo, it was made into a movie in 1972. It stars Marlon Brando and Al Pacino as the leaders of a fictional New York crime family.

If Vito Corleone was a QA engineer these are things he may say:

GodFather QA Quotes

Great Test Made

"Great tests are not born great, they grow great . . ."

No matter how good a test case is, there's always room to make it better. Don't assume that once you create a test case, that it will be perfect forever. There's going to change the may make the test run faster, test better or fix issues discovered by exploratory testing.

Trust Automation

"I don't trust automation to protect us, I have no intention of placing my fate in the hands of machines whose only qualification is that they managed to con a block of code to test for them."
- Mario Puzo, The Godfather

Automation is critical to every test plan. However, it would be a mistake to let all the testing be done by automation. QA should run some manual critical path testing to make sure that the product is working as expected.

In my experience, I have found that automation may miss some critical errors and not raise an alert to the automation review team.

Behind Testing

"Behind every successful test action there is a bug."
- Mario Puzo, The Godfather

No matter how good a test is, bugs will still get by QA. The important thing is for QA to find as many customer-facing bugs as possible. Realistically all bugs won't get found. (Some QA bugs may not get fixed because of low priority issues.)

November 25, 2019

IBM - You Make the Call

It's a slow week in QA, so I thought I put up an original QA graphic.

You Make the Call

If you were watching the NFL in the 1980s, you probably saw the famous IBM commercials where they challenged you to make the call. These commercials would show some strange plays and let the viewer think on what the correct call should be.

I reference that phrase, "You Make the Call" all the time to the engineering and operations team. So, I decided to create an updated graphic.

You Make The Call

You can catch some of the classic IBM commercials on YouTube:

November 18, 2019

GitHub Tips for QA

Here are four useful tips on using GitHub for QA:

Tags

If your company use tags, you should know about the Tags project page. It displays the latest tag and get the ability to download the tags.

https://github.com/project/tags

Releases

Find out the latest release and past releases with dates of deployment on the releases page. Not every project have project "releases" but those that do may have useful information here.

https://github.com/project/releases

KeyBoard Shortcuts

Learn some of the popular Keyboard shortcuts that are available on GitHub

Shift ?

Today I Learn: The fastest way to any GitHub Repository Wiki is to type in G W on any Repository page.

Use the Show All to view all the cool short cut options. (or simply go to the Keyboard shortcuts - GitHub Help page.)

View the Blame on Any File

Using the information from the previous tip, when looking at a GitHub file, if you type in b - you'll get modification information on that file. Perfect when you are debugging an issue and need to talk to someone about why a change was made.

One More Thing

Have you noticed that the Github search graphic is a perfect QA graphic:

Git Hub Search

I don't think it's realistic that people will be searching for bugs on GitHub.

November 11, 2019

Mandela Effect

Have you heard of the Mandela Effect?

Here's a brief description:

This is a "False Memory Phenomenon" where many people believe in a false memory. For example, Frona Broome had a false memory that Nelson Mandela died in the 1980s. That thought was shared by thousands of other people.

So how's does this relate to QA activities?

In release testing, people may have a false memory of features being tested before signing off on the current release. Everyone may assume that a feature was tested because it always is.

Note: This could also be true with automation. A feature that people may think is covered by automation is in fact still in development.

Value of Having a Checklist

This is why having a product-related checklist is critical to ensuring that all critical paths are tested. Don't just assume that someone else has tested it - find out by asking!

Reset the checklist for every release - that way you can ensure that critical features have been touched by manual or automation.

November 4, 2019

Best Screen Capture Tool for Chrome

When it comes to screen capture applications there are a lot of different options. You certainly don't need to buy one as all the major operating systems now have built-in screen capture tools.

If you constantly take screen shots of webpages in Google Chrome there's one tool that stands out above the rest. That's Snag-it.

A "Must Have" Tool for QA

What makes Snag-it the best screen capture tool for Chrome? Well, there are things that I can think of.:

  • Snag-it will automatically capture the inner window browser frame area. This means that you can capture the main webpage area. You don't need to play around guessing the window area.
  • Snag-it can capture the whole browser window. So if you need to show the URL in your screenshot you can.
  • Snag-it can scroll and capture the full webpage. This is useful when you need to capture the whole page in a bug report or for archive purposes.
  • You can select multiple regions. This is great when you have two Chrome browsers open and you're doing a comparison. You can also have shown the page against a product spec and highlight areas that are incorrect.
  • You can capture both images and video using the browser region functionality.
  • You can drag and drop a Bookmark icon to the Snagit Menu Bar icon, or to the Dock icon to quickly get a full website capture. Downside: If your site requires a login you would only see the login page. Still a neat way to capture a URL without having to load it in a browser.
  • When you capture the website the URL is saved as part of the MetaData. You can view the content using the "Capture Info" under Effect Styles. Not only will you get the URL, but Snagit will display the Google Chrome version and the date/time of capture. All useful information in any bug report.

Try it!

You can try out SnagiT for free for 15-days from TechSmith's website: Download Free Trial. After that the cost is $49.95.

What do you think?

Is there another tool that offers better screen capture functionality? Let me know what I am missing in the comments below.

October 28, 2019

Purpose of Smoke Testing

Smoke Testing is a critical part of Quality Assurance Testing. It's usually done to make sure that a deployment was done successfully. (Such as Code Freeze or Production Release)

When considering putting together a Smoke Test plan for you Software as a Service (SaS) application, here are some things to think about.

Five Things to Consider in Smoke Testing

  1. Critical path - Smoke test should focus on the critical path that customers will take to use your application.
  2. Focus on Making the Test Quick - The goal of smoke testing it to make sure they run fast. There shouldn't be complex steps to make sure the software works. Depending on the application, the smoke test should be completed in 20-minutes. Otherwise your simply running a modified version of a regression test.
  3. Logs should be open - Keep the Console logs open when running the test. (People do forget this!) This way you can catch errors as test are being run - such as tracking servers throwing 404s.
  4. Touch features that are most likely to break - Some Test should be testing complex part of the application that is likely to break - or have broken in the past. Examples might be the login process for existing users or uploading large files.
  5. Special Test for Production - In production smoke test, focus on features that are specific to production - such as load balance servers testing - as you may not have that in the test environment.

Test Early and Often

These are just some of the things that I think about when putting together an effective smoke test plan.

What are some of the things that you consider when forming a Smoke Test strategy?

October 21, 2019

What is Testing?

Explain Testing Chalk

Last week someone Tweeted this:

Explain testing without using the word testing.

Seems like an easy challenge:

The job of a Quality Assurance Engineer is to make sure that the product delivered works exceptionally for customers.

Customer can be any group: paying customers, prospects, employees - you get the idea.

Alternatively it could be written as:

A group of engineers that constantly challenge the Engineering team that code complete is working as per specification.

October 14, 2019

Best QA Advice

On Twitter, someone asked, "What's the best testing advice you've ever heard?" I thought it was an interesting question, so I thought I share my "Top 5 QA Advice" that I have gotten over the years.

Best QA Advice

Top 5 QA Advice

Here are some of the best advice I have learned over the years.

"Never assume the assumption is true." - Kind of weird advice. But it was given to me when I was assuming too much when reading the testing steps of a ticket. My manager at the time said this to me so that I would try to think outside of the box and challenge the assumption that was given on the ticket. (For example, what if the user forgot the email address that they used in a "Forgotten Password" test.)

Pay attention to the details - Always make sure to check the little things when testing tickets. Because when customers are looking at the final work, they will notice the small details - especially if it doesn't seem quite right.

Learn SQL really well - Given to me by a manager a long time ago. He really stressed to make sure that I knew all the tricks of making a SQL Select statement. This not only helped me debug issues but also build some interesting QA Dashboards.

Document Everything - The manager would go on to say, "You never know when you may get hit by a bus." He was being funny. But his point was to make sure that test documents were updated and more importantly useful. This was a key thing. If I wasn't writing things down.

Always end an automation run with a validation statement. - Someone on the automation team gave me this advice when code auditing my automation. "What good is your test if you don't finish it with some validation statement." This is pretty good advice that I give to other people.

October 7, 2019

QA Off Cycle Duties

Usually the QA team is busy testing. However, there are times during the sprint cycle where there is nothing to test.

This isn’t a break for QA. Rather it’s a shift in QA duties.

Eight Things QA Does in the Off Cycle

A Combination of tasks that QA can do when there's "nothing to do."

Test Case Repository - Always be contributing to the test case repository. It's critical for QA to keep the content fresh and accurate.

Documentation - Update Wiki pages on testing techniques and test plans. Make sure that the Wiki has the most up to date information. Some things to think about: Critical Path Testing, Local Functionality Smoke Testing, Production Account information, Release Schedule, Staffing and so much more.

Review Previous Releases Testing - Self evaluation of how testing could have been done better. Think of ways that bugs and issues could have been caught early in the test cycle.

Performance Testing - How are typical tasks being performed in Production? Are new releases helping the performance of the application or hurting it? Review the findings with the product team on a frequent bases.

Load Testing - Is the website working well when it’s being hit 1,000 times? There are a lot of tools and techniques on how to do load testing without impacting customers.

Error Log Monitoring - Are there any errors that are being caught in various log tracking alerts?

Domain Knowledge - what’s going on in the industry. Are there any testing trends that QA Bloggers are talking about?

Audit Testing Functionality - What testing techniques can be done more efficiently using a different set of tools. If you haven't changed your testing tools in a year - your running on an old version of the QA Operating System. (A play of words on why Apple updates their OS every year.)