How save or post data to firebase database application using python

Tutorial about how to use firebase database to store and read data using python

Thanks for the Amit Kumar youtube channel on making this tutorial.

Advertisements

Usage of Python Requests Library

First things first, let’s introduce you to Requests.

What is the Requests Resource?

Requests is an Apache2 Licensed HTTP library, written in Python. It is designed to be used by humans to interact with the language. This means you don’t have to manually add query strings to URLs, or form-encode your POST data. Don’t worry if that made no sense to you. It will in due time.

What can Requests do?

Requests will allow you to send HTTP/1.1 requests using Python. With it, you can add content like headers, form data, multipart files, and parameters via simple Python libraries. It also allows you to access the response data of Python in the same way.

In programming, a library is a collection or pre-configured selection of routines, functions, and operations that a program can use. These elements are often referred to as modules, and stored in object format.

Libraries are important, because you load a module and take advantage of everything it offers without explicitly linking to every program that relies on them. They are truly standalone, so you can build your own programs with them and yet they remain separate from other programs.

Think of modules as a sort of code template.

To reiterate, Requests is a Python library.

How to Install Requests

The good news is that there are a few ways to install the Requests library. To see the full list of options at your disposal, you can view the official install documentation for Requests here.

You can make use of pip, easy_install, or tarball.

If you’d rather work with source code, you can get that on GitHub, as well.

For the purpose of this guide, we are going to use pip to install the library.

In your Python interpreter, type the following:

pip install requests

Importing the Requests Module

To work with the Requests library in Python, you must import the appropriate module. You can do this simply by adding the following code at the beginning of your script:

import requests

Of course, to do any of this – installing the library included – you need to download the necessary package first and have it accessible to the interpreter.

Making a Request

When you ping a website or portal for information this is called making a request. That is exactly what the Requests library has been designed to do.

To get a webpage you would do something like the following:

r = request.get(‘https://github.com/timeline.json’)

Working with Response Code

Before you can do anything with a website or URL in Python, it’s a good idea to check the current status code of said portal. You can do this with the dictionary look-up object.

r = requests.get('https://github.com/timeline.json')
r.status_code
>>200
 
r.status_code == requests.codes.ok
>>> True
 
requests.codes['temporary_redirect']
>>> 307
 
requests.codes.teapot
>>> 418
 
requests.codes['o/']
>>> 200

Get the Content

After a web server returns a response, you can collect the content you need. This is also done using the get requests function.

import requests
r = requests.get('https://github.com/timeline.json')
print r.text
 
# The Requests library also comes with a built-in JSON decoder,
# just in case you have to deal with JSON data
 
import requests
r = requests.get('https://github.com/timeline.json')
print r.json

Working with Headers

By utilizing a Python dictionary, you can access and view a server’s response headers. Thanks to how Requests works, you can access the headers using any capitalization you’d like.

If you perform this function but a header doesn’t exist in the response, the value will default to None.

r.headers
{
    'status': '200 OK',
    'content-encoding': 'gzip',
    'transfer-encoding': 'chunked',
    'connection': 'close',
    'server': 'nginx/1.0.4',
    'x-runtime': '148ms',
    'etag': '"e1ca502697e5c9317743dc078f67693f"',
    'content-type': 'application/json; charset=utf-8'
}
 
r.headers['Content-Type']
>>>'application/json; charset=utf-8'
 
r.headers.get('content-type')
>>>'application/json; charset=utf-8'
 
r.headers['X-Random']
>>>None
 
# Get the headers of a given URL
resp = requests.head("http://www.google.com")
print resp.status_code, resp.text, resp.headers

Encoding

Requests will automatically decade any content pulled from a server. But most Unicode character sets are seamlessly decoded anyway.

When you make a request to a server, the Requests library make an educated guess about the encoding for the response, and it does this based on the HTTP headers. The encoding that is guessed will be used when you access the r.text file.

Through this file, you can discern what encoding the Requests library is using, and change it if need be. This is possible thanks to the r.encoding property you’ll find in the file.

If and when you change the encoding value, Requests will use the new type so long as you call r.text in your code.

print r.encoding
>> utf-8
 
>>> r.encoding = ‘ISO-8859-1’

Custom Headers

If you want to add custom HTTP headers to a request, you must pass them through a dictionary to the headers parameter.

import json
url = 'https://api.github.com/some/endpoint'
payload = {'some': 'data'}
headers = {'content-type': 'application/json'}
 
r = requests.post(url, data=json.dumps(payload), headers=headers)

Redirection and History

Requests will automatically perform a location redirection when you use the GET and OPTIONS verbs in Python.

GitHub will redirect all HTTP requests to HTTPS automatically. This keeps things secure and encrypted.

You can use the history method of the response object to track redirection status.

r = requests.get('http://github.com')
r.url
>>> 'https://github.com/'
 
r.status_code
>>> 200
 
r.history 
>>> []

Make an HTTP Post Request

You can also handle post requests using the Requests library.

r = requests.post(http://httpbin.org/post)

But you can also rely on other HTTP requests too, like PUTDELETEHEAD, and OPTIONS.

r = requests.put("http://httpbin.org/put")
r = requests.delete("http://httpbin.org/delete")
r = requests.head("http://httpbin.org/get")
r = requests.options("http://httpbin.org/get")

You can use these methods to accomplish a great many things. For instance, using a Python script to create a GitHub repo.

import requests, json
 
github_url = "https://api.github.com/user/repos"
data = json.dumps({'name':'test', 'description':'some test repo'})
r = requests.post(github_url, data, auth=('user', '*****'))
 
print r.json

Errors and Exceptions

There are a number of exceptions and error codes you need to be familiar with when using the Requests library in Python.

  • If there is a network problem like a DNS failure, or refused connection the Requests library will raise a ConnectionError exception.
  • With invalid HTTP responses, Requests will also raise an HTTPError exception, but these are rare.
  • If a request times out, a Timeout exception will be raised.
  • If and when a request exceeds the preconfigured number of maximum redirections, then a TooManyRedirects exception will be raised.

Any exceptions that Requests raises will be inherited from the requests.exceptions.RequestException object.

You can read more about the Requests library at the links below.

http://docs.python-requests.org/en/latest/api/

http://pypi.python.org/pypi/requests

http://docs.python-requests.org/en/latest/user/quickstart/

http://isbullsh.it/2012/06/Rest-api-in-python/#requests

What is __pycache__ in python?

Explanation 1:

When you run a program in python, the interpreter compiles it to bytecode first (this is an oversimplification) and stores it in the __pycache__ folder. If you look in there you will find a bunch of files sharing the names of the .py files in your project’s folder, only their extensions will be either .pyc or .pyo. These are bytecode-compiled and optimized bytecode-compiled versions of your program’s files, respectively.

As a programmer, you can largely just ignore it… All it does is make your program start a little faster. When your scripts change, they will be recompiled, and if you delete the files or the whole folder and run your program again, they will reappear (unless you specifically suppress that behavior)

If you are using cpython (which is the most common, as it’s the reference implementation) and you don’t want that folder, then you can suppress it by starting the interpreter with the -B flag, for example

python -B foo.py

Another option, as noted by tcaswell, is to set the environment variable PYTHONDONTWRITEBYTECODE to any value (according to python’s man page, any “non-empty string”).

Explanation 2:

__pycache__ is a folder containing Python 3 bytecode compiled and ready to be executed.

Don’t bother deleting these files nor suppress creation. It’s pointless and will actually hurt performance.

Python programmers usually ignore bytecode. Indeed __pycache__ and *.pyc are common lines to see in .gitignore files. Bytecode is not meant for distribution and can be disassembled using dis module.

Explanation 3:

A __pycache__folder is created when you use the line

import file_name

or try to get information from another file you have created. This makes it a little faster when running the program your second time to open the other file.

Difference between iterator, iterable, and iteration in python

iterable = [1, 2] 

iterator = iter(iterable)

print(iterator.__next__())   

print(iterator.__next__())   

so,

  1. iterable is an object that can be looped over. e.g. list , string , tuple etc.
  2. using the iter function on our iterable object will return an iterator object.
  3. now this iterator object has method named __next__ (in Python 3, or just next in Python 2) by which you can access each element of iterable.

so, output of the above code will be:

1

2

Iteration is a general term for taking each item of something, one after another. Any time you use a loop, explicit or implicit, to go over a group of items, that is iteration

 

Python explanation: print “%20s” % “Hello World”

In following code:
bar = "Hello World"
print("%5s" % bar)

Total width of bar should be more than 5 characters otherwise padding spaces will be added as prefix.

Here padding is 5 but string length is 11. So nothing will happen.

In following code:

bar = "Hello World"
print("%15s" % bar)

Padding is 15 which exceeds string length 11. Hence 4 spaces will be added in beginning.

Output will be: ----Hello World

- denotes one space.

Header : MIME types

The Multipurpose Internet Mail Extensions (MIME) type is a standardized way to indicate the nature and format of a document. It is defined and standardized in IETF RFC 6838. The Internet Assigned Numbers Authority (IANA) is the official body responsible for keeping track of all official MIME types, and you can find the most up-to-date and complete list at the Media Types page.

Browsers often use the MIME type (and not the file extension) to determine how it will process a document; it is therefore important that servers are set up correctly to attach the correct MIME type to the header of the response object.

 

Syntax

General structure

type/subtype

The structure of a MIME type is very simple; it consists of a type and a subtype, two strings, separated by a '/'. No space is allowed. The type represents the category and can be a discrete or a multipart type. The subtype is specific to each type.

A MIME type is case-insensitive but traditionally is written all in lower case.

Discrete types

text/plain
text/html
image/jpeg
image/png
audio/mpeg
audio/ogg
audio/*
video/mp4
application/*
application/json
application/javascript
application/ecmascript
application/octet-stream
…

Discrete types indicate the category of the document, it can be one of the following:

Type Description Example of typical subtypes
text Represents any document that contains text and is theoretically human readable

text/plain, text/html,

 text/css,

text/javascript

image Represents any kind of images. Videos are not included, though animated images (like animated gif) are described with an image type. image/gif, image/png,

image/jpeg, image/bmp,

image/webp

audio Represents any kind of audio files audio/midi,

audio/mpeg,

audio/webm,

audio/ogg,

audio/wav

video Represents any kind of video files

video/webm,

video/ogg

application Represents any kind of binary data. application/octet-stream,

application/pkcs12,

application/vnd.mspowerpoint,

application/xhtml+xml,

application/xml,

application/pdf

Multipart types

multipart/form-data
multipart/byteranges

Multipart types indicate a category of document that are broken in distinct parts, often with different MIME types. It is a way to represent a composite document. With the exception of multipart/form-data, that are used in relation of HTML Forms and POSTmethod, and multipart/byteranges that are used in conjunction with 206 Partial Content status message to send only a subset of a whole document, HTTP doesn’t handle multipart documents in a specific way: the message is simply transmitted to the browser (which will likely propose a Save As window, not knowing how to display the document inline.)

Important MIME types for Web developers

application/octet-stream

This is the default value for a binary file. As it really means unknown binary file, browsers usually don’t automatically execute it, or even ask if it should be executed. They treat it as if the Content-Disposition header was set with the value attachment and propose a ‘Save As’ file.

text/plain

This is the default value for textual files. Even if it really means unknown textual file, browsers assume they can display it.

Note that text/plain does not mean any kind of textual data. If they expect a specific kind of textual data, they will likely not consider it a match. Specifically if they download a text/plain file from a <link> element declaring a CSS files, they will not recognize it as a valid CSS files if presented with text/plain. The CSS mime type text/css must be used.

text/css

Any CSS files that have to be interpreted as such in a Web page must be of the text/cssfiles. Often servers do not recognize files with the .css suffix as CSS files, instead they send them with text/plain or application/octet-stream MIME type: in these cases, they won’t be recognized as CSS files by most browsers and will be silently ignored. Special attention has to be paid to serve CSS files with the correct type.

text/html

All HTML content should be served with this type. Alternative MIME types for XHTML (like application/xml+html) are mostly useless nowadays (HTML5 unified these formats).

Images types

Only a handful of image types are widely recognized and are considered Web safe, ready for use in a Web page:

MIME type Image type
image/gif GIF images (lossless compression, superseded by PNG)
image/jpeg JPEG images
image/png PNG images
image/svg+xml SVG images (vector images)

There is a discussion to add WebP (image/webp) to this list, but as each new image type will increase the size of a codebase, this may introduce new security problems, so browser vendors are cautious in accepting it.

Other kinds of images can be found in Web documents. For example, many browsers support icon image types for favicons or similar. In particular, ICO images are supported in this context with the image/x-icon MIME type.

Audio and video types

Like images, HTML doesn’t define a set of supported types to use with the <audio>and<video> elements, so only a relatively small group of them can be used on the Web. The Media formats supported by the HTML audio and video elements explains both the codecs and container formats which can be used.

The MIME type of such files mostly represent the container formats and the most common ones in a Web context are:

MIME type Audio or video type
audio/wave
audio/wav
audio/x-wav
audio/x-pn-wav
An audio file in the WAVE container format. The PCM audio codec (WAVE codec “1”) is often supported, but other codecs have more limited support (if any).
audio/webm An audio file in the WebM container format. Vorbis and Opus are the most common audio codecs.
video/webm A video file, possibly with audio, in the WebM container format. VP8 and VP9 are the most common video codecs used within it; Vorbis and Opus the most common audio codecs.
audio/ogg An audio file in the OGG container format. Vorbis is the most common audio codec used in such a container.
video/ogg A video file, possibly with audio, in the OGG container format. Theora is the usual video codec used within it; Vorbis is the usual audio codec.
application/ogg An audio or video file using the OGG container format. Theora is the usual video codec used within it; Vorbis is the usual audio codec.

multipart/form-data

The multipart/form-data type can be used when sending the content of a completed HTML Form from the browser to the server. As a multipart document format, it consists of different parts, delimited by a boundary (a string starting with a double dash '--'). Each part is an entity by itself, with its own HTTP headers, Content-Disposition, and Content-Type for file uploading fields, and the most common (Content-Length is ignored as the boundary line is used as the delimiter).

Content-Type: multipart/form-data; boundary=aBoundaryString
(other headers associated with the multipart document as a whole)

--aBoundaryString
Content-Disposition: form-data; name="myFile"; filename="img.jpg"
Content-Type: image/jpeg

(data)
--aBoundaryString
Content-Disposition: form-data; name="myField"

(data)
--aBoundaryString
(more subparts)
--aBoundaryString--

The following form:

<form action="http://localhost:8000/" method="post" enctype="multipart/form-data">
  <input type="text" name="myTextField">
  <input type="checkbox" name="myCheckBox">Check</input>
  <input type="file" name="myFile">
  <button>Send the file</button>
</form>

will send this message:

POST / HTTP/1.1
Host: localhost:8000
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:50.0) Gecko/20100101 Firefox/50.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Connection: keep-alive
Upgrade-Insecure-Requests: 1
Content-Type: multipart/form-data; boundary=---------------------------8721656041911415653955004498
Content-Length: 465

-----------------------------8721656041911415653955004498
Content-Disposition: form-data; name="myTextField"

Test
-----------------------------8721656041911415653955004498
Content-Disposition: form-data; name="myCheckBox"

on
-----------------------------8721656041911415653955004498
Content-Disposition: form-data; name="myFile"; filename="test.txt"
Content-Type: text/plain

Simple file.
-----------------------------8721656041911415653955004498--

multipart/byteranges

The multipart/byteranges MIME type is used in the context of sending partial responses back to the browser. When the 206 Partial Content status code is sent, this MIME type is used to indicate that the document is composed of several parts, one for each of the requested range. Like other multipart types, the Content-Type uses the boundary directive to define the boundary string. Each of the different parts have a Content-Type header with the actual type of the document and a Content-Range with the range they represent.

HTTP/1.1 206 Partial Content
Accept-Ranges: bytes
Content-Type: multipart/byteranges; boundary=3d6b6a416f9b5
Content-Length: 385

--3d6b6a416f9b5
Content-Type: text/html
Content-Range: bytes 100-200/1270

eta http-equiv="Content-type" content="text/html; charset=utf-8" />
    <meta name="vieport" content
--3d6b6a416f9b5
Content-Type: text/html
Content-Range: bytes 300-400/1270

-color: #f0f0f2;
        margin: 0;
        padding: 0;
        font-family: "Open Sans", "Helvetica
--3d6b6a416f9b5--

Importance of setting the correct MIME type

Most web servers send unknown-type resources using the default application/octet-stream MIME type. For security reasons, most browsers do not allow setting a custom default action for such resources, forcing the user to store it to disk to use it. Some commonly seen incorrect server configurations happen with the following file types:

  • RAR-encoded files. In this case, the ideal would be to set the true type of the encoded files; this is often not possible (as it may not be known to the server and these files may contain several resources of different types). In this case, configuring the server to send the application/x-rar-compressed MIME type, users will not have defined a useful default action for them.
    • Audio and video files. Only resources with the correct MIME Type will be recognized and played in <video> or <audio> elements. Be sure to use the correct type for audio and video.
    • Proprietary file types. Pay particular attention when serving a proprietary file type. Avoid using application/octet-stream as special handling will not be possible: most browsers do not allow defining a default behavior (like “Opening in Word”) for this generic MIME type.
  • MIME sniffing

    In the absence of a MIME type, or in some other cases where a client believes they are incorrectly set, browsers may conduct MIME sniffing, which is guessing the correct MIME type by looking at the resource. Each browser performs this differently and under different circumstances. There are some security concerns with this practice, as some MIME types represent executable content and others not. Servers can block MIME sniffing by sending the X-Content-Type-Options along the Content-Type.

    Other methods of conveying document type

    MIME types are not the only way to convey the document type information:

Sensor calibration and importance of it

Sensor calibration is a method of improving sensor performance by removing structural errors in the sensor outputs. Structural errors are differences between a sensors expected output and its measured output, which show up consistently every time a new measurement is taken.

What is the purpose of the calibration?

Purpose of instrument calibration. Calibration refers to the act of evaluating and adjusting the precision and accuracy of measurement equipment. Instrument calibration is intended to eliminate or reduce bias in an instrument’s readings over a range for all continuous values.

How do you calibrate a temperature sensor?

Method 1: Ice Water

  1. Fill a glass with ice cubes, then top off with cold water.
  2. Stir the water and let sit for 3 minutes.
  3. Stir again, then insert your thermometer into the glass, making sure not to touch the sides.
  4. The temperature should read 32°F (0°C). Record the difference and offset your thermometer as appropriate.

Why is it important to calibrate?

Calibration is a comparison between a known measurement (the standard) and the measurement using your instrument. Typically, the accuracy of the standard should be ten times the accuracy of the measuring device being tested. However, accuracy ratio of 3:1 is acceptable by most standards organizations.

What is a relative import? In what are places star import was allowed in python 2.7 and python 3 with examples

Relative import happens whenever you are importing a package relative to the current script/package.

Consider the following tree for example:

mypkg
├── base.py
└── derived.py

Now, your derived.py requires something from base.py. In Python 2, you could do it like this (in derived.py):

from base import BaseThing

Python 3 no longer supports that since it’s not explicit whether you want the ‘relative’ or ‘absolute’ base. In other words, if there was a Python package named base installed in the system, you’d get the wrong one.

Instead it requires you to use explicit imports which explicitly specify location of a module on a path-alike basis. Your derived.py would look like:

from .base import BaseThing

The leading . says ‘import base from module directory’; in other words, .base maps to ./base.py.

Similarly, there is .. prefix which goes up the directory hierarchy like ../ (with ..mod mapping to ../mod.py), and then ... which goes two levels up (../../mod.py) and so on.

Please however note that the relative paths listed above were relative to directory where current module (derived.py) resides in, not the current working directory.


Explanation about the star import case

For example, you need to use a few math functions but you use them only in a single function. In Python 2 you were permitted to be semi-lazy:

def sin_degrees(x):
    from math import *
    return sin(degrees(x))

Note that it already triggers a warning in Python 2:

a.py:1: SyntaxWarning: import * only allowed at module level
  def sin_degrees(x):

In modern Python 2 code you should and in Python 3 you have to do either:

def sin_degrees(x):
    from math import sin, degrees
    return sin(degrees(x))

or:

from math import *

def sin_degrees(x):
    return sin(degrees(x))

Tips to Debug the import Issues in python

There are basically 3 tips I have for debugging import issues:

  1. Use the interactive interpreter (The REPL) to import the modules and see if you are getting what you expect.
  2. Start your script with python -v -m my_scriptname.py and then check the output to see exactly where your modules are getting imported from. (-m represents modular initialization)
  3. Use Pycharm. Pycharm’s fantastic introspection abilities mean that you will immeadiately know whether or not your module is being properly imported as it will indicate an error if not. It will sometimes also suggest the proper correction.

    Note: The community edition is free and if you’re a student you can get a free subscription to ALL of their products!