Overview

Whenever we call a generator function, it returns an iterator which is known as a generator. But what is a generator function?  A generator function is just a normal function, but whenever it generates a value, it makes use of the yield keyword rather than return. Yield expression pauses the function to save the current state and later resumes from the last saved state. Whenever we use yield expression in the def function body it makes that function a generator and using it in the async def function’s body causes that coroutine function to be an asynchronous generator function. The generator function returns an object which is known as a generator iterator when being called.

Code:

def gen(): #generator function
	yield ‘normal generator’
	yield ‘multiple yield’

g = gen()
print(g) #<generator object gen at 0x0000001AC35895E70>
print(next(g)) #normal generator
print(g.__next__()) #multiple yield
print(next(g)) #StopIteration

Here, next is used for advancing and it also returns any yielded item, which is the same as g.__next__(). Here next(g) will throw an error (StopIteration) if there are no more elements to advance further.

Generator Expression

Generator expressions are like list comprehensions, but they return a generator instead of a list. In list comprehensions we use [square bracket] but here we use (parenthesis).

Code:

gen_exp = (i for i in range(20))
print(gen_exp) #<generator object <genexpr> at 0x0000001D5B4995E70>
print(next(gen_exp)) #0
print(next(gen_exp)) #1
print(sum(gen_exp)) #189

#Very very fast for handling large sequences as compared to list comprehensions
gen_exp2 = (i for i in range(1000000000000000000))
for i in range(10):
	print(next(gen_exp2)) #0,1,2,3,4,5,6,7,8,9

Introduction of yield from 

The addition of yield breaks complex processing into smaller pieces when processing lines in a file.

Code:

#Without using yield from
def logProcessing(file):
	for line in open(file, "r"):
		yield line

#Using yield from
def logProcessing(file):
	yield from open(file, "r")

Connecting Generators

Yield from can also be helpful in chaining generators together.

Code:

def first(a):
	for i in range(a):
		yield i

def second(a, b):
	for j in range(b):
		yield j

def third(a, b):
	yield from first(a)
	yield from second(a, b)
	yield from second(a, b)

print(list(first(5))) #[0,1,2,3,4]
print(list(second(5, 10))) #[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
print(list(third(2, 8))) #[0, 1, 0, 1, 2, 3, 4, 5, 6, 7, 0, 1, 2, 3, 4, 5, 6, 7]

When to use Generators

• Processing log files or large data: Generators can be very handy in stream processing as the generation of value only happens when needed. It also leads to low CPU utilization as well as performance improvement.
• Piping: They can also be used for chaining multiple generators.
• Concurrency: Whenever you have asynchronous functions, you can use generators to switch between multiple generators when one is not in use while the other can be used.

Generating Fibonacci sequence using Generator

Code:

def  fib(inp):
"""Generator to evaluate fibonacci sequence"""
	first, second = 0, 1
	for i in range(inp):
		yield first
		first, second = second, first + second
	inp = 10 #input
	print(list(fib(inp))) #[0, 1, 1, 2, 3, 5, 8, 13, 21, 34]

Special thanks to Arya Pratap Singh for contributing to this article on takeUforward. If you also wish to share your knowledge with the takeUforward fam, please check out this article. If you want to suggest any improvement/correction in this article please mail us at [email protected]