read(), readline(), readlines() & xreadlines() func in Python

While doing a regression analysis, I had to read & parse through a large .dat file. I called readline() func but was surprised to see that I was actually iterating through every single character in the first line!

with open(filePath, 'r') as f:     
for line in f.readline():
print line

Then realization dawned upon me!

* read(size) >> size is an optional numeric argument and this func returns a quantity of data equal to size. If size if omitted, then it reads the entire file and returns it

size − This is the number of bytes to be read from the file.

* readline() >> reads a single line from file with newline at the end

* readlines() >> returns a list containing all the lines in the file

* xreadlines() >> Returns a generator to loop over every single line in the file

Advertisements

What is Pyroelectricity?

Pyroelectricity is the ability of certain materials to generate an electrical potential when they are heated or cooled.

As a result of this change in temperature, positive and negative charges move to opposite ends through migration (i.e. the material becomes polarised) and hence, an electrical potential is established.

Pyroelectricity can be visualized as one side of a triangle, where each corner represents energy states in the crystal: kinetic, electrical and thermal energy.

The side between electrical and thermal corners represents the pyroelectric effect and produces no kinetic energy.

The side between kinetic and electrical corners represents the piezoelectric effect and produces no heat.

Although artificial pyroelectric materials have been engineered, the effect was first discovered in minerals such as quartz and tourmaline and other ionic crystals.

Read and Write From Serial Port With Raspberry Pi

n this tutorial we will see how to use the serial port on Raspberry Pi. We will use the serial port available on Raspberry with a RS232/TTL 3-5,5V adapter and a USB-serial adapter. By default the Raspberry Pi’s serial port is configured to be used for console input/output. This can help to fix problems during boot, or to log in to the Pi if the video and network are not available.

To be able to use the serial port to connect and talk to other devices (e.g. a modem a printer.. ), the serial port console login needs to be disabled.

Here we use Raspberry Pi 2, and we connect a RS232/TTL 3-5,5V adapter to pins 4 (5V), 6 (GND) ,8 (TX),10 (RX) of Raspberry, obviously connect tx with rx and vice versa.

Emmeshop tutorial Raspberry serial connection

To search for available serial ports we use the command

dmesg | grep tty

The output is something like this

pi@raspberrypi ~ $ dmesg | grep tty
  [    0.000000] Kernel command line: dma.dmachans=0x7f35 bcm2708_fb.fbwidth=656 bcm2708_fb.fbheight=416 bcm2709.boardrev=0xa01041 bcm2709.serial=0x93f9c7f9 smsc95xx.macaddr=B8:27:EB:F9:C7:F9 bcm2708_fb.fbswap=1 bcm2709.disk_led_gpio=47 bcm2709.disk_led_active_low=0 sdhci-bcm2708.emmc_clock_freq=250000000 vc_mem.mem_base=0x3dc00000 vc_mem.mem_size=0x3f000000  dwc_otg.lpm_enable=0 console=tty1 console=ttyAMA0,115200 root=/dev/mmcblk0p2 rootfstype=ext4 elevator=deadline rootwait
  [    0.001774] console [tty1] enabled
  [    0.749509] dev:f1: ttyAMA0 at MMIO 0x3f201000 (irq = 83, base_baud = 0) is a PL011 rev3
  [    1.268971] console [ttyAMA0] enabled
  pi@raspberrypi ~ $

Last line indicates that the console is enabled on the serial port ttyAMA0, so we disable it

Run the configuration command and follow the instructions below

sudo raspi-config

Emmeshop tutorial Raspberry serial

Emmeshop tutorial Raspberry serial

Emmeshop tutorial Raspberry serial

Emmeshop tutorial Raspberry serial

Emmeshop tutorial Raspberry serial

Reboot and try with

dmesg | grep tty

output now is

pi@raspberrypi ~ $ dmesg | grep tty
  [    0.000000] Kernel command line: dma.dmachans=0x7f35 bcm2708_fb.fbwidth=656 bcm2708_fb.fbheight=416 bcm2709.boardrev=0xa01041 bcm2709.serial=0x93f9c7f9 smsc95xx.macaddr=B8:27:EB:F9:C7:F9 bcm2708_fb.fbswap=1 bcm2709.disk_led_gpio=47 bcm2709.disk_led_active_low=0 sdhci-bcm2708.emmc_clock_freq=250000000 vc_mem.mem_base=0x3dc00000 vc_mem.mem_size=0x3f000000  dwc_otg.lpm_enable=0 console=tty1 root=/dev/mmcblk0p2 rootfstype=ext4 elevator=deadline rootwait
  [    0.001769] console [tty1] enabled
  [    0.749438] dev:f1: ttyAMA0 at MMIO 0x3f201000 (irq = 83, base_baud = 0) is a PL011 rev3
  pi@raspberrypi ~ $

Now we can use the serial ttyAMA0. We connect an adapter usb / serial, then we will try to establish a communication between the two serial ports; obviously in a practical application to every serial we could connect a device, for example a modem, a printer a RFID reader etc.

Emmeshop tutorial Raspberry two serial port communication

After connecting the second serial port we launch the command to find the name that Raspberry gives him

dmesg | grep tty

The output is something like this

pi@raspberrypi ~ $ dmesg | grep tty
  [    0.000000] Kernel command line: dma.dmachans=0x7f35 bcm2708_fb.fbwidth=656 bcm2708_fb.fbheight=416 bcm2709.boardrev=0xa01041 bcm2709.serial=0x93f9c7f9 smsc95xx.macaddr=B8:27:EB:F9:C7:F9 bcm2708_fb.fbswap=1 bcm2709.disk_led_gpio=47 bcm2709.disk_led_active_low=0 sdhci-bcm2708.emmc_clock_freq=250000000 vc_mem.mem_base=0x3dc00000 vc_mem.mem_size=0x3f000000  dwc_otg.lpm_enable=0 console=tty1 root=/dev/mmcblk0p2 rootfstype=ext4 elevator=deadline rootwait
  [    0.001769] console [tty1] enabled
  [    0.749438] dev:f1: ttyAMA0 at MMIO 0x3f201000 (irq = 83, base_baud = 0) is a PL011 rev3
  [  971.919417] usb 1-1.2: pl2303 converter now attached to ttyUSB0
  pi@raspberrypi ~ $

Ok, now we create two files, one who writes something on the ttyAMA0 port and the other that reads on the ttyUSB0 port.

serial_write.py

    #!/usr/bin/env python
          
      
           import time
           import serial
          
      
           ser = serial.Serial(
              
               port='/dev/ttyAMA0',
               baudrate = 9600,
               parity=serial.PARITY_NONE,
               stopbits=serial.STOPBITS_ONE,
               bytesize=serial.EIGHTBITS,
               timeout=1
           )
           counter=0
          
      
           while 1:
               ser.write('Write counter: %d \n'%(counter))
               time.sleep(1)
               counter += 1

serial_read.py

    #!/usr/bin/env python
          
      
           import time
           import serial
          
      
           ser = serial.Serial(
              
               port='/dev/ttyUSB0',
               baudrate = 9600,
               parity=serial.PARITY_NONE,
               stopbits=serial.STOPBITS_ONE,
               bytesize=serial.EIGHTBITS,
               timeout=1
           )
           counter=0
          
      
           while 1:
               x=ser.readline()
               print x

If we run both files, serial_read.py will read what serial_write.py writes

Emmeshop tutorial Raspberry serial write and read

This is just a small example but it can serve as a starting point to send a print to an old printer or read data from a router or a gps.

AWS Cannot download a root CA for IoT

Reader Query: 

When I’m trying to create a new certificate – I can see the page with 4 downloading links (keys and certificates).
Unfortunately, I can download all, except for root CA, because when I’m pushing the last – it throws me to AWS Documentation.

Solution:

The AWS documentation that the link is redirecting you to contains the Root CA certificates. The Root CA certificates are available under the “Server Authentication” section:
https://docs.aws.amazon.com/iot/latest/developerguide/managing-device-certs.html#server-authentication

Based on the signing key, the following certificates are available from the documentation page:

-> RSA 2048 bit key: VeriSign Class 3 Public Primary G5 root CA certificate (https://www.symantec.com/content/en/us/enterprise/verisign/roots/VeriSign-Class%203-Public-Primary-Certification-Authority-G5.pem)
-> RSA 2048 bit key: Amazon Root CA 1 (https://www.amazontrust.com/repository/AmazonRootCA1.pem)
-> RSA 4096 bit key: Amazon Root CA 2 (https://www.amazontrust.com/repository/AmazonRootCA2.pem)
-> ECC 256 bit key: Amazon Root CA 3 (https://www.amazontrust.com/repository/AmazonRootCA3.pem)
-> ECC 384 bit key: Amazon Root CA 4 (https://www.amazontrust.com/repository/AmazonRootCA4.pem)

If you are creating a new thing using the web console and downloading the credentials (device certificate, device public and private key, etc) then “Amazon Root CA 1” can be used as the root CA certificate.
While creating a new thing from the console,  please do not forget to click on the “Activate” button prior to proceeding to the “Attach a Policy” section.

Difference between GPIO.setmode(BCM) and GPIO.setmode(BOARD)

It tells the library which pin nunbering system you are going to use. BOARD signifies using the physical pin numbers on the RPi P1 connector. BCM signifies the Broadcom SOC channel designation. One word of warning – the BCM channels moved around a little between revision 1 and revision 2 of the Raspberry Pi board. The BOARD numbering system stays working between board revisions.

What is Metadata with easy examples

Metadata is simply data about data. It means it is a description and context of the data. It helps to organize, find and understand data. Here are a few real world examples of metadata:

Typical metadata

Those are some typical metadata elements:

  1. Title and description,
  2. Tags and categories,
  3. Who created and when,
  4. Who last modified and when,
  5. Who can access or update.

A photo

Every time you take a photo with today’s cameras a bunch of metadata is gathered and saved with it:

  • date and time,
  • filename,
  • camera settings,
  • geolocation.

A book

Each book has a number of standard metadata on the covers and inside. This includes:

  • a title,
  • author name,
  • publisher and copyright details,
  • description on a back,
  • table of contents,
  • index,
  • page numbers.

A blog post

Every blog post has standard metadata fields that are usually at before first paragraph. This includes:

  • title,
  • author,
  • published time,
  • category,
  • tags.

Email

Every email you send or receive has a number of metadata fields, many of which are hidden in the message header and not visible to you in your mail client. This metadata includes:

  • subject,
  • from,
  • to,
  • date and time sent,
  • sending and receiving server names and IPs,
  • format (plain text of HTLM),
  • anti-spam software details.

Word document

Every word processing software collects some standard metadata and enables you to add your own fields for each document. Typical fields are:

  • title,
  • subject,
  • author,
  • company,
  • status,
  • creation date and time,
  • last modification date and time,
  • number of pages.

A spreadsheet

Spreadsheets contain a few metadata fields:

  • tab names,
  • table names,
  • column names,
  • user comments.

Relational database

Relational databases (most common type of database) store and provide access not only data but also metadata in a structure called data dictionary or system catalog. It holds information about:

  • tables,
  • columns,
  • data types,
  • constraints
  • table relationships,
  • and many more

Document and share database metadata

Extract metadata from databases, document schema and share across your orgranization in convenient HTML with Dataedo.

Computer files

All the fields you see by each file in file explorer is actually metadata. The actual data is inside those files. Metadata includes:

  • file name,
  • type,
  • size,
  • creation date and time,
  • last modification date and time.

Web page

Every web page has a number of metadata fields:

  • page title,
  • page description,
  • icon.

Paper files

Paper document files have often administrative metadata that help manage documents. This might include:

  • letter for files organized alphabetically,
  • access control information (“classified” for instance),
  • logos.

Raspberry Pi Vs Intel Edison Features Comparison

HR-OS1 means?

The HR-OS1 is powered by an onboard Linux computer giving you all the processing power you need to run your robot. The HR-OS1 is a hackable, modular, humanoid robot development platform designed from the ground up with customization and modification in mind.

Raspberry Pi Vs Intel Edison

Your HR-OS1 kit shipped with 2 choices for the Main CPU – the Intel Edison and the Raspberry Pi. Both platforms will run the same HR-OS1 framework to control your robot. Each platform has its strong points, as outlined below.

#Intel Edison Pros

  • Power Consumption The Intel Edison consumes far less power than the Raspberry Pi.
  • Size Even with the breakout board, the Intel Edison is much smaller then the Raspberry Pi.
  • Built in Wifi/Bluetooth The Intel Edison does not need extra dongles to add bluetooth / wifi functionality.
  • Built in FTDI USB-Serial Converter You can connect the Intel Edison Mini Breakout directly to you computer to open a serial terminal to control the Edison.
  • More Mature Development/Testing We have been working with the Intel Edison for much longer than the Raspberry Pi, so the framework is better tested with the Intel Edison.

#Raspberry Pi Pros

  • USB Ports The Raspberry Pi has 4 available USB ports to attach devices to.
  • SD Card Slot The Raspberry Pi’s SD based storage makes it possible to upgrade your disk size. Each HR-OS1 kit ships with an 8GB SD card, compared to the Edison’s built in 4gb eMMC storage.
  • Easy access GPIO pins The Raspberry Pi has 3.3v I/O pins that are easily accessible through the male pin headers. The Intel Edison has 1.8v pins.
  • Standard Ports HDMI, ethernet, composite video, and audio ports
  • **Special Ports ** Camera and display interface ports
  • 3d Graphics core
  • Superior processor