What is a relative import? In what are places star import was allowed in python 2.7 and python 3 with examples

Relative import happens whenever you are importing a package relative to the current script/package.

Consider the following tree for example:

mypkg
├── base.py
└── derived.py

Now, your derived.py requires something from base.py. In Python 2, you could do it like this (in derived.py):

from base import BaseThing

Python 3 no longer supports that since it’s not explicit whether you want the ‘relative’ or ‘absolute’ base. In other words, if there was a Python package named base installed in the system, you’d get the wrong one.

Instead it requires you to use explicit imports which explicitly specify location of a module on a path-alike basis. Your derived.py would look like:

from .base import BaseThing

The leading . says ‘import base from module directory’; in other words, .base maps to ./base.py.

Similarly, there is .. prefix which goes up the directory hierarchy like ../ (with ..mod mapping to ../mod.py), and then ... which goes two levels up (../../mod.py) and so on.

Please however note that the relative paths listed above were relative to directory where current module (derived.py) resides in, not the current working directory.


Explanation about the star import case

For example, you need to use a few math functions but you use them only in a single function. In Python 2 you were permitted to be semi-lazy:

def sin_degrees(x):
    from math import *
    return sin(degrees(x))

Note that it already triggers a warning in Python 2:

a.py:1: SyntaxWarning: import * only allowed at module level
  def sin_degrees(x):

In modern Python 2 code you should and in Python 3 you have to do either:

def sin_degrees(x):
    from math import sin, degrees
    return sin(degrees(x))

or:

from math import *

def sin_degrees(x):
    return sin(degrees(x))
Advertisements

Tips to Debug the import Issues in python

There are basically 3 tips I have for debugging import issues:

  1. Use the interactive interpreter (The REPL) to import the modules and see if you are getting what you expect.
  2. Start your script with python -v -m my_scriptname.py and then check the output to see exactly where your modules are getting imported from. (-m represents modular initialization)
  3. Use Pycharm. Pycharm’s fantastic introspection abilities mean that you will immeadiately know whether or not your module is being properly imported as it will indicate an error if not. It will sometimes also suggest the proper correction.

    Note: The community edition is free and if you’re a student you can get a free subscription to ALL of their products!

What is __init__.py used for?

Explanation 1

The primary use of __init__.py is to initialize Python packages. The easiest way to demonstrate this is to take a look at the structure of a standard Python module.

package/
    __init__.py
    file.py
    file2.py
    file3.py
    subpackage/
        __init__.py
        submodule1.py
        submodule2.py

As you can see in the structure above the inclusion of the __init__.py file in a directory indicates to the Python interpreter that the directory should be treated like a Python package

What goes in __init__.py?

__init__.py can be an empty file but it is often used to perform setup needed for the package(import things, load things into path, etc).

One common thing to do in your __init__.py is to import selected Classes, functions, etc into the package level so they can be convieniently imported from the package.

In our example above we can say that file.py has the Class File. So without anything in our __init__.py you would import with this syntax:

from package.file import File

However you can import File into your __init__.py to make it available at the package level:

# in your __init__.py
from file import File

# now import File from package
from package import File

Another thing to do is at the package level make subpackages/modules available with the __all__ variable. When the interpeter sees an __all__ variable defined in an __init__.pyit imports the modules listed in the __all__ variable when you do:

from package import *

__all__ is a list containing the names of modules that you want to be imported with import * so looking at our above example again if we wanted to import the submodules in subpackage the __all__ variable in subpackage/__init__.py would be:

__all__ = ['submodule1', 'submodule2']

With the __all__ variable populated like that, when you perform

from subpackage import *

it would import submodule1 and submodule2.

Explanation 2

What is a Python package?

A Python package is simply an organized collection of python modules. A python module is simply a single python file.

Creating a package with __init__.py is all about making it easier to develop l

Why would I want to create a package using __init__.py?

arge Python projects. It provides an easy way for you to group large folders of many seperate python scripts into a single importable module.

Let’s run through some examples

The best way to understand why you would use __init__.pyand to learn how to use it to create a package is to run through some quick examples!

The code in this tutorial should work for Python 2 or 3. Just remember, if you are using 2 then you will need to use the from __future__ import print_function functionality.

Say we have three modules we have created:

someFolder
|-- stringLength.py
|-- stringToUpper.py
`-- stringToLower.py

(Remember a module is considered to be any single python file)

Let’s say the content of these files is the following:

# stringLength.py

def stringLength(inStr):
    return len(inStr)
# stringToUpper.py

def stringToUpper(inStr):
    return inStr.upper()
# stringToLower.py

def stringToLower(inStr):
    return inStr.lower()

Obviously, these functions are useless, but it helps to serve as a model for the basic concept that we have some python modules that we have already written that are somehow related.

So, without creating a package and using __init__.py, how do we use the functions in these files?

Well, we can only import these files if they are in the current directory that whatever script we are running is running from.

Well, we can use these files in a new Python script but with one key caveat:

  • The files must be in the same directory as the script we are trying to use them in.

To illustrate that, let’s create a file called example1.py that leverages our modules:

# example1.py

import stringLength
import stringToLower
import stringToUpper

some_string = "Hello, Universe!"

print(stringLength.stringLength(some_string))
print(stringToLower.stringToLower(some_string))
print(stringToUpper.stringToUpper(some_string))

Adding a blank __init__.py

What if we wanted to seperate these scripts into a folder in order to keep them more organized?

Well, that is where the __init__.py file comes into play.

First, lets move our scripts into a new subfolder and call it: string_func. Then create an empty file in that folder called __init__.py

Here is our new file/folder structure:

someFolder
|-- string_func
|   |-- __init__.py
|   |-- stringToUpper.py
|   |-- stringToLower.py
|   `-- strengthLength.py
`-- example1.py

So, now let’s test out exactly what __init__.py allows us to do:

Let’s make a new example2.py file.

# example2.py

import string_func.stringLength
import string_func.stringToLower
import string_func.stringToUpper

some_string = "Hello, Universe!"

print(string_func.stringLength.stringLength(some_string))
print(string_func.stringToLower.stringToLower(some_string))
print(string_func.stringToUpper.stringToUpper(some_string))

So, now we can access our string functions in this manner. This is great, because they are all in a seperate folder, but the syntax is definitely not very succinct. Let’s see if we can clean things up a bit by editing our __init__.py file.

Adding imports to init.py

Open your __init__.py file and make the following changes:

# __init__.py
from .stringLength import stringLength
from .stringToLower import stringToLower
from .stringToUpper import stringToUpper

# Explanation about the init import below
# from .stringLength(stringLength.py file) import stringLength function

Note that the . before the module name is neccessary as of Python 3 since it is more strict regarding relative imports: https://stackoverflow.com/questions/12172791/changes-in-import-statement-python3?utm_medium=organic&utm_source=google_rich_qa&utm_campaign=google_rich_qa

And so with that in our __init__.py we can now shorten our code to:

# example3.py

import string_func

some_string = "Hello, Universe!"

print(string_func.stringLength(some_string))
print(string_func.stringToLower(some_string))
print(string_func.stringToUpper(some_string))

Now the syntax is a lot shorter and you can see that string_func is behaving like its own module.

So, that is basically what __init__.py does! It allows you to treat a directory as if it was a python module. Then you can further define imports inside your __init__.py file to make imports more succinct, or you can just leave the file blank.

For more information about python modules and packages you can see check the python documentation on it.

As you can see __init__.py can be very useful besides its primary function of indicating that a directory is a module. If you have any comments or questions, hit up the comments

 

 

Python cyclic error solution- Explained with example

What is a Circular Dependency?

A circular dependency occurs when two or more modules depend on each other. This is due to the fact that each module is defined in terms of the other (See Figure 1).

For example:

functionA():  
    functionB()

And

functionB():  
    functionA()

The code above depicts a fairly obvious circular dependency. functionA() calls functionB(), thus depending on it, and functionB() calls functionA(). This type of circular dependency has some obvious problems, which we’ll describe a bit further in the next section.

Figure 1

Problems with Circular Dependencies

Circular dependencies can cause quite a few problems in your code. For example, it may generate tight coupling between modules, and as a consequence, reduced code reusability. This fact also makes the code more difficult to maintain in the long run.

In addition, circular dependencies can be the source of potential failures, such as infinite recursions, memory leaks, and cascade effects. If you’re not careful and you have a circular dependency in your code, it can be very difficult to debug the many potential problems it causes.

What is a Circular Import?

Circular importing is a form of circular dependency that is created with the import statement in Python.

For example, let’s analyze the following code:

# module1
import module2

def function1():  
    module2.function2()

def function3():  
    print('Goodbye, World!')
# module2
import module1

def function2():  
    print('Hello, World!')
    module1.function3()
# __init__.py

import module1

module1.function1()  

When Python imports a module, it checks the module registry to see if the module was already imported. If the module was already registered, Python uses that existing object from cache. The module registry is a table of modules that have been initialized and indexed by module name. This table can be accessed through sys.modules.

If it was not registered, Python finds the module, initializes it if necessary, and executes it in the new module’s namespace.

In our example, when Python reaches import module2, it loads and executes it. However, module2 also calls for module1, which in turn defines function1().

The problem occurs when function2() tries to call module1’s function3(). Since module1 was loaded first, and in turn loaded module2 before it could reach function3(), that function isn’t yet defined and throws an error when called:

$ python __init__.py
Hello, World!  
Traceback (most recent call last):  
  File "__init__.py", line 3, in <module>
    module1.function1()
  File "/Users/scott/projects/sandbox/python/circular-dep-test/module1/__init__.py", line 5, in function1
    module2.function2()
  File "/Users/scott/projects/sandbox/python/circular-dep-test/module2/__init__.py", line 6, in function2
    module1.function3()
AttributeError: 'module' object has no attribute 'function3'  

How to Fix Circular Dependencies

In general, circular imports are the result of bad designs. A deeper analysis of the program could have concluded that the dependency isn’t actually required, or that the depended functionality can be moved to different modules that wouldn’t contain the circular reference.

A simple solution is that sometimes both modules can just be merged into a single, larger module. The resulting code from our example above would look something like this:

# module 1 & 2

def function1():  
    function2()

def function2():  
    print('Hello, World!')
    function3()

def function3():  
    print('Goodbye, World!')

function1()  

However, the merged module may have some unrelated functions (tight coupling) and could become very large if the two modules already have a lot code in them.

So if that doesn’t work, another solution could have been to defer the import of module2 to import it only when it is needed. This can be done by placing the import of module2 within the definition of function1():

# module 1

def function1():  
    import module2
    module2.function2()

def function3():  
    print('Goodbye, World!')

In this case, Python will be able to load all functions in module1 and then load module2 only when needed.

This approach doesn’t contradict Python syntax, as the Python documentation says: “It is customary but not required to place all import statements at the beginning of a module (or script, for that matter)”.

The Python documentation also says that it is advisable to use import X, instead of other statements, such as from module import *, or from module import a,b,c.

You may also see many code-bases using deferred importing even if there isn’t a circular dependency, which speeds up the startup time, so this is not considered bad practice at all (although it may be bad design, depending on your project).

Wrapping up

Circular imports are a specific case of circular references. Generally, they can be resolved with better code design. However, sometimes, the resulting design can contain a large amount of code, or mix unrelated functionalities (tight coupling).

Have you run in to circular imports in your own code? If so, how did you fix it? Let us know in the comments!