How good is ChatGPT at programming?
These days, it seems, every developer is test-driving — and then writing about — the code-generating capabilities of OpenAI’s ChatGPT. Admittedly, I also was curious about the quality of code ChatGPT could generate. So, I too threw my hat in the ring, explored ChatGPT’s seemingly uncanny abilities and wrote about it.
Really though, my thoughts ran along the lines of whether ChatGPT would make us software developers redundant in the next five years. Many commentators do rate the code it writes. It was time to find out what ChatGPT could do.
FIRST ITERATION
I asked ChatGPT to create a ShoppingCart class[1]:
Prompt:
In C#, write a ShoppingCart class and associated classes for an eCommerce
website. Ideally, this code will also be reusable in other systems.
Desired behaviour:
1. Must be able to add items to the shopping cart.
2. Each item represents a quantity of product (e.g. 1 Apple, 5 Bananas, etc.)
3. Must be able to remove items from the shopping cart.
4. Must be able to clear out (i.e. empty) all items from a shopping cart.
5. A shopping cart can tell us all the items' total amount or price.
6. We can get the collection of items in the shopping cart.And here is what it generated:
Answer:
using System.Collections.Generic;
public class ShoppingCart
{
private List<Product> products;
public ShoppingCart()
{
products = new List<Product>();
}
public void AddItem(Product product)
{
products.Add(product);
}
public void RemoveItem(Product product)
{
products.Remove(product);
}
public void Clear()
{
products.Clear();
}
public double TotalAmount()
{
double total = 0;
foreach (Product product in products)
{
total += product.Price * product.Quantity;
}
return total;
}
public List<Product> GetItems()
{
return products;
}
}
public class Product
{
public int Quantity { get; set; }
public string Name { get; set; }
public double Price { get; set; }
public Product(int quantity, string name, double price)
{
Quantity = quantity;
Name = name;
Price = price;
}
}I was impressed, especially with the ShoppingCart class; ChatGPT had created all the asked-for methods, and the logic contained in them appeared correct too. In particular, the TotalAmount() implementation is non-trivial. Well done, ChatGPT! Parameterless methods such as TotalAmount() and GetItems() could have been improved by producing them as public read-only properties.
ChatGPT also generated a Product class to act as the container for the product Name (e.g. “Apple” or “Toyota Corolla”), the Price, and the number of product items (i.e. Quantity).
A few critiques came to mind regarding class Product:
- The Price property name is a bit ambiguous. It represents the price for one of the products — a unit price. UnitPrice would have been a clearer descriptor.
- Using the double data type for monetary amounts may lead to precision errors when multiplying Quantity by (unit) Price to get the total amount. The decimal data type would be better for Price.
- I had the preconceived thought that with my prompt of “each item represents a quantity of product“, ChatGPT would make a Product as well as a CartItem class. I was wrong. Class Product contains Quantity too. This is a bit strange — 3 Apples is not a different product from 5 Apples. The product is Apples — a thing without a quantity. From a modelling perspective, we can do better than ChatGPT here: Product ought to encapsulate the product name and the unit price, while the product and quantity belong in a different class, say, CartItem.
- By default, class Product is mutable, with public setters for everything. I wasn’t too happy about that. Ideally, objects should be immutable, or at least minimally mutable.
Can you do better, ChatGPT? It was up to level up and improve my prompt.
SECOND ITERATION
What did I want to change this time around?
- To reduce object mutability, I included the phrase “use clean code principles where appropriate“.
- I changed my phraseology to get ChatGPT to see that we needed a CartItem and a Product class.
Prompt:
In C#, write a ShoppingCart class and associated classes for an eCommerce
website. This code should be reusable in other systems. Use clean code
principles where appropriate.
Desired behaviours:
1. Must be able to add items to the shopping cart.
2. Each item in the shopping cart is represented by a product and a quantity (e.g. 1 Apple, 5 Bananas, etc.)
3. Must be able to remove items from the shopping cart.
4. Must be able to clear out (i.e. empty) all items from a shopping cart.
5. A shopping cart can tell us the total amount or price of all the items in it.
6. We can get the collection of items in the shopping cart.Answer:
using System.Collections.Generic;
public class ShoppingCart
{
private List<CartItem> items;
public ShoppingCart()
{
items = new List<CartItem>();
}
public void AddItem(Product product, int quantity)
{
var item = items.Find(i => i.Product.Id == product.Id);
if (item == null)
{
items.Add(new CartItem(product, quantity));
}
else
{
item.Quantity += quantity;
}
}
public void RemoveItem(Product product)
{
var item = items.Find(i => i.Product.Id == product.Id);
if (item != null)
{
items.Remove(item);
}
}
public void Clear()
{
items.Clear();
}
public double TotalAmount()
{
double total = 0;
foreach (var item in items)
{
total += item.Product.Price * item.Quantity;
}
return total;
}
public IEnumerable<CartItem> GetItems()
{
return items;
}
}
public class CartItem
{
public Product Product { get; set; }
public int Quantity { get; set; }
public CartItem(Product product, int quantity)
{
Product = product;
Quantity = quantity;
}
}
public class Product
{
public int Id { get; set; }
public string Name { get; set; }
public double Price { get; set; }
public Product(int id, string name, double price)
{
Id = id;
Name = name;
Price = price;
}
}The impact of my improved prompting was immediate:
- Yes! ChatGPT generated classes for CartItem and Product. Each contained the right fields in the right place: Product had Name, and Price, and — nicely enough — also a new Id field. ChatGPT has correctly determined that class Product needed an identity. Wow! I didn’t expect that. Class CartItem contains field references to the Product class (tick) and the Quantity (also tick).
- On the other hand, my ‘clean code’ hint must have been too subtle; ChatGPT infused Product and CartItem with property setters for all getters. What do I have to do for ChatGPT to get with the program? I don’t want to spell it out, or I might as well do it myself.
ShoppingCart had changed for the better too:
- Each ShoppingCart instance would now hold a collection of CartItems, not Products.
- When adding or removing an item, the code would check whether the product already existed in the cart — by product id. Impressive! However, for top marks, the repeated logic to find the item would need to have been encapsulated in a separate private helper method to maximise readability for us mere humans.
- I particularly liked how ChatGPT generated AddItem() so that the cart's product quantities are combined (added). E.g. If we have 3 Apples already in the cart and add another 2 Apples, then we end up with 5 Apples.
THIRD ITERATION
I was annoyed that TotalAmount() and GetItems() were not being generated as public properties. Furthermore, TotalAmount(), as a method did not contain a verb. What do you do method TotalAmount()??
Fair enough — ChatGPT needs a hint to produce the cart’s TotalAmount and Items collections as properties.
Prompt:
In C#, write a ShoppingCart class and associated classes for an eCommerce
website. This code should be reusable in other systems. Use clean code
principles where appropriate.
Desired behaviours:
1. Must be able to add items to the shopping cart.
2. Each item in the shopping cart is represented by a product and a quantity (e.g. 1 Apple, 5 Bananas, etc.)
3. Must be able to remove items from the shopping cart.
4. Must be able to clear out (i.e. empty) all items from a shopping cart.
5. A shopping cart can tell us the total amount or price of all the items in it, as a property.
6. We can get the collection of items in the shopping cart, as a property.ChatGPT did this.
Answer:
using System.Collections.Generic;
public class ShoppingCart
{
private List<CartItem> _items;
public ShoppingCart()
{
_items = new List<CartItem>();
}
public void AddItem(Product product, int quantity)
{
var item = _items.Find(i => i.Product.Id == product.Id);
if (item == null)
{
_items.Add(new CartItem(product, quantity));
}
else
{
item.Quantity += quantity;
}
}
public void RemoveItem(Product product)
{
var item = _items.Find(i => i.Product.Id == product.Id);
if (item != null)
{
_items.Remove(item);
}
}
public void Clear()
{
_items.Clear();
}
public double TotalAmount
{
get
{
double total = 0;
foreach (var item in _items)
{
total += item.Product.Price * item.Quantity;
}
return total;
}
}
public IEnumerable<CartItem> Items
{
get { return _items; }
}
}
public class CartItem
{
public Product Product { get; set; }
public int Quantity { get; set; }
public CartItem(Product product, int quantity)
{
Product = product;
Quantity = quantity;
}
}
public class Product
{
public int Id { get; set; }
public string Name { get; set; }
public double Price { get; set; }
public Product(int id, string name, double price)
{
Id = id;
Name = name;
Price = price;
}
}Nice. ChatGPT got the hint and created public properties TotalAmount and Items, respectively.
FOURTH ITERATION
So far, so good. Next, let’s ask ChatGPT to generate a (hopefully complete) suite of unit tests.
Prompt:
In C#, write a ShoppingCart class and associated classes for an eCommerce
website. This code should be reusable in other systems. Use clean code
principles where appropriate. Also write a complete set of XUnit unit tests
for the ShoppingCart's behaviours.
Desired behaviours:
1. Must be able to add items to the shopping cart.
2. Each item in the shopping cart is represented by a product and a quantity (e.g. 1 Apple, 5 Bananas, etc.)
3. Must be able to remove items from the shopping cart.
4. Must be able to clear out (i.e. empty) all items from a shopping cart.
5. A shopping cart can tell us the total amount or price of all the items in it, as a property.
6. We can get the collection of items in the shopping cart, as a property.Answer:
using System.Collections.Generic;
public class Product
{
public int Id { get; set; }
public string Name { get; set; }
public decimal Price { get; set; }
public Product(int id, string name, decimal price)
{
Id = id;
Name = name;
Price = price;
}
}
public class CartItem
{
public Product Product { get; set; }
public int Quantity { get; set; }
public CartItem(Product product, int quantity)
{
Product = product;
Quantity = quantity;
}
}
public class ShoppingCart
{
private List<CartItem> _items;
public IReadOnlyCollection<CartItem> Items = _items.AsReadOnly();
public ShoppingCart()
{
_items = new List<CartItem>();
}
public void AddItem(Product product, int quantity)
{
var item = _items.Find(i => i.Product.Id == product.Id);
if (item == null)
{
_items.Add(new CartItem(product, quantity));
}
else
{
item.Quantity += quantity;
}
}
public void RemoveItem(Product product)
{
var item = _items.Find(i => i.Product.Id == product.Id);
if (item != null)
{
_items.Remove(item);
}
}
public void Clear()
{
_items.Clear();
}
public double TotalAmount
{
get
{
double total = 0;
foreach (var item in _items)
{
total += item.Product.Price * item.Quantity;
}
return total;
}
}
}Interestingly, ChatGPT decided to switch the Product’s Price to the more suitable decimal data type at this point. Could this be because it started setting values into a product’s price in the unit tests below?
Unit Tests:
public class ShoppingCartTests
{
private readonly Product _product1 = new Product(1, "Apple", 0.5m);
private readonly Product _product2 = new Product(2, "Banana", 0.1m);
[Fact]
public void AddItem_Could_Add_New_Item_To_Cart()
{
var cart = new ShoppingCart();
cart.AddItem(_product1, 5);
Assert.Single(cart.Items);
Assert.Equal(5, cart.Items.First().Quantity);
Assert.Equal(_product1, cart.Items.First().Product);
}
[Fact]
public void AddItem_Could_Increase_Quantity_For_Existing_Item()
{
var cart = new ShoppingCart();
cart.AddItem(_product1, 5);
cart.AddItem(_product1, 3);
Assert.Single(cart.Items);
Assert.Equal(8, cart.Items.First().Quantity);
Assert.Equal(_product1, cart.Items.First().Product);
}
[Fact]
public void RemoveItem_Could_Remove_Item_From_Cart()
{
var cart = new ShoppingCart();
cart.AddItem(_product1, 5);Unfortunately, ChatGPT only produced a partial test suite — a couple of tests and a half-completed one. I’m not sure why. Maybe I’d reached an output character limit?
Conclusion
- ChatGPT is producing impressive-looking code. Simple algorithms, like calculating the total amount for the contents of our ShoppingCart class, are logically correct.
- However, for all the nice code, ChatGPT is not checking input data for validity. Arguably, I didn’t explicitly prompt for input data validation. Yet, even when input values are clearly invalid — for example, negative or zero product quantities — ChatGPT just sat on the sidelines twiddling its virtual thumbs. It completely overlooked that adding 0 of anything to a shopping cart doesn't make sense. This scenario is invalid, and ChatGPT should have disallowed it, but the AI did not pick up on this wrinkle. A human developer would likely have noticed.
- Nonetheless, I see myself increasingly using ChatGPT in my daily programming activities. Especially for repetitive tasks. I also know, though, that I cannot mindlessly rely on ChatGPT to get it right all the time. I must check what’s been produced.
If you enjoyed this article, please leave some claps — and a bunch of claps if you loved it! :) Thank you kindly.
Join my email list to fast-track your software engineering career.
When signing up, you’ll get my guide, ‘The Road to Master Progammer’, containing 3 powerful ideas to help you shorten your journey to expert programmer.
Footnotes:
[1] I had intended to include the original screenshots of my prompts and ChatGPT’s answers, rather than reproducing the text. However, these ended up being too low resolution for a good viewing experience. When I tried to redo the same prompts, ChatGPT had already picked up some new tricks. Maybe we need to worry about our human programming jobs after all?!?
