How to Remove Duplicates from a List in Python

Lists in Python are a super convenient way to store a lot of information in a single variable.

In a list, items are held in individual indexes, and you can access them by calling the index number.

However, because you determine how items are added, it can contain duplicate values.

In this post, we'll learn the two best ways to remove duplicates from a list in Python.

How to remove duplicates from a list in Python using a Set

The easiest way to remove duplicates from a list is to throw all items into the set, the convert the set back into a list.

The reason this works is because sets are unordered collections of unique items, meaning it will ignore any duplicates.

Let's start with an example list:

	numbers = [1, 2, 3, 3, 4, 5, 5, 6, 7, 8, 9, 9]

print(numbers)

The output will be:

	[1, 2, 3, 3, 4, 5, 5, 6, 7, 8, 9, 9]

Clearly, this list contains duplicates.

Let's create a set and then convert the set back:

	numbers = [1, 2, 3, 3, 4, 5, 5, 6, 7, 8, 9, 9]

numbers = set(numbers)
numbers = list(numbers)

print(numbers)

	[1, 2, 3, 4, 5, 6, 7, 8, 9]

How to remove duplicates from a list in Python using a Dictionary

The cool thing about dictionaries is that their keys have to be unique.

This makes sense because you can't have two keys with the same name or else you wouldn't be able to access one of them.

This means that we can make a dictionary from a list, then just return the keys.

Using our earlier example:

	numbers = [1, 2, 3, 3, 4, 5, 5, 6, 7, 8, 9, 9]

print(numbers)

	[1, 2, 3, 3, 4, 5, 5, 6, 7, 8, 9, 9]

We can use dict.fromKeys() to create a dictionary from the list. Then we just need to take those keys and convert them back into a list.

	numbers = [1, 2, 3, 3, 4, 5, 5, 6, 7, 8, 9, 9]

numbers = dict.fromkeys(numbers)
numbers = list(numbers)

print(numbers)

	[1, 2, 3, 4, 5, 6, 7, 8, 9]

Conclusion

In this post, we learned how to remove duplicates from a list in Python.

We learned that the easiest way to do this is to convert the list into a set, then convert the set back into a list, but you can also use a dictionary and convert the keys back into a list.

Thanks for reading and happy coding!

• Support Us
• Join
• Share
• Tweet
• Share