Friday, October 29, 2010

Image Alignment and Stitching

MSR Richard Szeliski:
http://research.microsoft.com/pubs/70092/tr-2004-92.pdf

Hugin - Panorama photo stitcher

Open Source:
http://hugin.sourceforge.net/

Wednesday, October 27, 2010

Single Camera Calibration and Stereo camera calibration

Parameters:

focal length, principal point, skew, distortion
rotation vector, translation vector


GML C++ Camera Calibration Toolbox

http://graphics.cs.msu.ru/en/science/research/calibration/cpp

MSR Zhang:
Zhang's camera calibration


Tsai Camera Calibration Software (fish eye distortion)

Camera Calibration Toolbox for Matlab
Microsoft Easy Camera Calibration Tool


Tina (calibration and computation)??
[1] N.A.Thacker and J.E.W.Mayhew. `Optimal Combination of Stereo
Camera Calibration from Arbitrary Stereo Images', Image and Vision
Computing. 9(1),27-32, 1990.

Determining the Epipolar Geometry and its Uncertainty: A Review


Camera calibration:
http://babel.isa.uma.es/mrpt/index.php/Downloads

Camera Calibration Toolbox for Matlab (Jean-Yves Bouguet)
Tangential distortion

Microsoft Easy Camera Calibration Tool

Image Stitching (camera calibration and image stitching are heavily used in Google maps)


Epipolar rectification


Traffic Photo Enhancement

http://www.photocop.com/recognition.htm

plate recognition

plate recognition is mature and well developed technology.

If you search it, you will find a lot:
http://www.platerecognition.info/1104.htm

Free and open source number plate recognition software
http://mac.softpedia.com/get/Utilities/JavaANPR.shtml

http://visl.technion.ac.il/projects/2003w24/

http://www.licenseplaterecognition.com/

A good introduction on the technology:

http://answers.google.com/answers/threadview/id/452412.html

Saturday, October 23, 2010

Real-Time Stereo Vision

Minoru 3D Webcam

http://nma.web.nitech.ac.jp/fukushima/minoru/minoru3D-e.html

Lambda Calculus and Functional Programming

Linux Software Raid 1 setup

Linux Software Raid 1 setup
http://www.linuxconfig.org/Linux_Software_Raid_1_Setup

Performance Analysis Tools for Linux Developers

http://www.drdobbs.com/open-source/220700195

Anaglyph Methods Comparison

Anaglyph Methods Comparison
http://www.3dtv.at/Knowhow/AnaglyphComparison_en.aspx

Computer Vision with OpenCV course

A nice step by step course Computer Vision with OpenCV course tutorial:
http://www.scss.tcd.ie/Darren.Caulfield/vision/

Monday, October 18, 2010

multithreading

Multithreading in Linux

mutex

mutex.hh


Critical Section

livelock

Multithreading in Windows

thread.hh
pthread.h


Pthread

pthread_setschedparam(pthread_self(), SCHED_OTHER, &sched);

struct Grabber_thread : Thread {

virtual void *thread_run()

io_thread.start();

thread safe

preemptive threading

context switch

event thread: causing the event thread to block for any length of time is considered bad programming practice. (unresponsive application)

Thread Synchronization

Monitors:

A monitor is a set of routines that are protected by a mutual exclusion lock.

A thread cannot execute any of the routines in the monitor until it acquires the lock, which means that only one thread at a time can execute within the monitor,
all other threads must wait for the currently executing thread to give up control of the lock.


Monitor is simpler to use.


Semaphores (mutex): A semaphore is a simpler construct, just a lock that protects a shared resource.

counting semaphores
event semaphore


Deadlock

Deadlock avoidance


Java:

public synchronized boolean deposit( double amount ) { }
public synchronized boolean withdraw( double amount) { }

or
synchronized(this) {


}

C#:

lock(this) {
.... //the code to protect, only within a method.
}


busy waiting

Java:

Thread task = new TheTask();

task.start();

while( task.isAlive() ) {
; // do nothing
}

Solution (shared object):

Object theLock = new Object(); //shared object

synchronized( theLock) {
Thread task = new TheTask( theLock );
task.start();


try {
theLock().wait();
}
catch (InterruptdException e) {
..... // do something if interrupted.
}
} //end of theLock


............

class TheTask extends Thread {

private Object theLock;

public TheTask( Object theLock) {
this.theLock = theLock;
}

public void run() {
synchronized ( theLock) {
....//do the task.
theLock.notify();
}
}
} //end of TheTask


///////////////////////////////////////////////////////
//The code can be simplified as:
///////////////////////////////////////////////////////

Thread task = new TheTask();
synchronized( task) {

task.start();

try {
task.wait();
}
catch ( InterruptedException e) {

.......//do something if interrupted
}

....................

class TheTask extends Thread {

public void run() {
synchronized( this ) {
.... // do the task.
this.notify();
}
}
} //end of TheTask


Producer/Consumer (java)

public class IntBuffer {

private int index;
private int[] buffer = new int[10];

public synchronized void add( int num ) {
while(index == buffer.length-1) {
try {
wait();
}
catch(InterruptedException e) {
}
} //end of while

buffer[index]=num; index++;
notifyAll();
}

public synchronized int remove() {
while(index == 0) {
try {
wait();
}
catch (InterruptedException e) {
}
} // end while

index --;
int ret = buffer[index];
notifyAll();
return(ret);

} //end of remove


} //end of class

public class Producer extends Thread {
private Intbuffer buffer;

public Producer( IntBuffer buffer) {
this.buffer = buffer;
}

public void run() {
Random r = new Random();
while(true) {
int num = r.nextInt();
buffer.add(num);
System.out.println("Produced " + num);
}
}
}


public class Consumer extends Thread {
private IntBuffer buffer;
public Consumer (IntBuffer buffer) {
this.buffer = buffer;
}

public void run() {
while (true) {
int num = buffer.remove();
System.out.println("Consumed " + num);
}
}
}


IntBuffer b = new IntBuffer();
Producer p = new Producer(b);
Consumer c = new Consumer(b);
p.start();
c.start();


The Dining Philosophers


public class DiningPhilosophers {

//each fork is just an Object we synchronize on
private Object[] forks;
private Philosopher[] philosophers;

private DiningPhilosophers (int num) {
forks = new Object[num];
philosophers = new philosopher[num];

for (int i = 0; i forks[i] = new Object();

int fork1 = i;
int fork2 = (i+1) % num;

if(i==0) //philosopher 0 picks up fork2 first.
philosophers[0] = new Philosopher( i, fork2, fork1 );
else

philosophers[i] = new Philosopher( i, fork1, fork2 );
}
}

public void startEating() throws InterruptedException {
for (int i =0; i < id="iv3o"> philosophers[i].start();
}

//suspend the main thread until the first philosopher stops eating, which will never happen
//this keeps the simulation running indefinitely

philosophers[0].join();
} //end startEating


private class Philosopher extends Thread {

private int id;
private int fork1;
private int fork2;

Philosopher (int id, int fork1, int fork2) {
this.id =id;
this.fork1 = fork1;
this.fork2 = fork2;
}

public void run() {

status("Ready to eat using forks "+fork1+ " and "+fork2);
pause(); //pause to let others get ready.

while(true) {
status("Picking up fork "+fork1);
synchronized ( forks[fork1]) {
status("Picking up fork "+fork2);
synchronized ( forks[fork2]) {
status("Eating");
} //fork2
} //fork1
} //while

} //end run

private void pause() {
try {
sleep(300);
}
catch ( InterruptedException e) {
//do nothing
}

}

private void status ( String msg) {
System.out.println("Philosopher " + id + ": " + msg);
}

} //end of philosopher

public static void main(String[] args) {
try {
DiningPhilosophers d = new DiningPhilosophers(5);
d.startEating();
}
catch ( InterruptedException e) {
}

} //end main

}


Multi-Threaded Programming With POSIX Threads(good tutorial)


Win32 Multithreaded Programming, Aaron Cohen and Mike Woodring, 1998
Errata and Source Code
Mike Woodring's .NET Sample Page

Three-Dimensional Computer Vision ---- A gentle introduction

Three-Dimensional Computer Vision

---- A gentle introduction

Liming Hu


Abstract-This paper is a brief introduction to the stereo vision from the perspective of geometry. I found that the papers and books I am reading about stereo vision are not so easy to understand for some one with basic image processing knowledge, so I wrote this introduction.

As shown in Fig. 1, the relationship between disparity and depth (also called range in some textbooks), r is the depth from the object P to the stereo camera head (camera C1 and C2 pairs), b, called baseline, is the horizontal distance between two camera head. The plane formed by points: P, C1, and C2 is called epipolar plane, the line M1M2 is called epipolar line, which is formed by the intersection of epipolar plane and the image plane, it is horizontal in Fig. 1. The corresponding match pixel M2 on the right image should lie in the epipolar line corresponding to pixel M1 on the left image, this is called epipolar constraint, which helps decrease the search space in the correspondence problem in stereo analysis.

Suppose the two cameras are the exactly the same:

They have the same focal length f (the distance from the center of the lens to the image plane), they have the same shutter speed, and take the pictures at the same time, and also the two cameras are put in the same plane, and mounted in parallel, they only have horizontal distance between them.

As shown in Fig. 1, dl is the horizontal distance from the image center O1 to the object image M1 for the left image; dr is the horizontal distance from the image center O2 to the object image M2 for the right image. The difference between dl and dr is called disparity, in Fig. 1, if the object falls between the two camera head, d = |dl + dr|. If the object P above the stereo camera head, is on left side or right side of the stereo head, then d = |dl - dr|, practically, because the distance between the two camera head is so small, the chance that the object can fall between the two camera head is low. In Fig. 1, it is straightforward that ΔP’PC1 and ΔP’PC1 are similar, so we have: b/r = d/f, it means that:

r=bf/d (1)

Eq. (1) means that depth is inverse proportion to the disparity. Suppose the distance between the two cameras is 12.57cm, and the focal length is 2.454mm (10-3 m), and the horizontal distance between two pixels in the image is 7µm (the reciprocal of the horizontal distance between two pixels in the image is called the scale in horizontal direction.), we have a disparity of 24 pixels (one pixel is interpolated to 16 subpixels), then the depth will be: 1.83m. As the depth increases, the disparity decreases faster, and finally it will be insensitive to the depth, it means change in disparity only takes place in a certain range of depth.

Figure 1. Relationship between disparity and depth

From Eq. (1), we can get:

әr/әd = -bf/d2 (2)

From Eq.(1), d = bf/r, substitute in Eq.(2), we can get:

әr/әd = - r2 /bf (3)

Suppose the origin of the coordination system is C1, we can get the coordinate of P by the similarity of ΔPO1’C1 and ΔM1O1C1:

x = -xl*b/d, y = -yl*b/d, z = f*b/d (4)

So it means stereo algorithm should search a certain range of disparities (disparity window), and the depth of objects can be determined by the disparity window is restricted to certain interval, called horopter. Horopter is very important in applications.

Horopter can be made large by: decreasing the baseline, decreasing the focal length (by using wider angle lenses, maybe severe distortion), increasing pixel width by down sampling the image, and increasing the disparity window. We should know that by decreasing the baseline, decreasing the focal length, and increasing pixel width to increase Horopter will have a side effect on decreasing the depth resolution as shown in Eq. (2).

Depth resolution is defined as minimal change in depth that the stereo camera head can differentiate. From Eq. (3), we can see, suppose that disparity resolution is given, the depth resolution increases (goes bad) as the baseline and focal length decreases, and the square of depth increases (so it means as the depth increases, the stereo camera will be less sensitive to the depth). Also we can see depth resolution is proportion to disparity resolution, so if the vision algorithm can interpolate the disparity images, then we can get better depth resolution, suppose disparities are interpolated to 1/16 pixels, so a search range of 24 (0~23) means that there are 24*16=384 disparity values (0~383), For the parameters given above, suppose at 1m depth, then әr = -12 * 7*10-6 / (12.57 * 10-2*2.454*10-3) = -2.27cm, at 3m depth, әr = -20.4cm, at 4m depth, әr = -36.3cm.

Depth resolution is different from range (depth) accuracy, range (depth) accuracy measures how well the computed range by the algorithm compares with the actual range (depth).

Some parameters that may influence stereo analysis result:

1) Confidence threshold: The confidence threshold will eliminate stereo matches that have a low probability of success because of lack of image texture; weak textures give a confidence measure below the threshold, and are eliminated by the algorithm. A good threshold value can be found by pointing the stereo cameras at a texture less surface, i.e. blank floor, and starting the stereo algorithm, start from the threshold of 0, until noise just disappears in the disparity image (the noises are replaced by a uniform black area.).

2) Correlation size: the size of the correlation window used for correspondence matching affects the result of disparity image; a larger window will produce smoother disparity images, and will miss smaller objects. A smaller window will give more spatial detail, but will tend to produce noisy disparity image.

Reference

[1] http://www.ai.sri.com/~konolige/svs

[2] Dhond, U.R. and Aggarwal, J.K. Structure from stereo-a review, IEEE Transactions on Systems, Man and Cybernetics, Vol. 19, No. 6, pp. 1489-1510, 1989.

[3] Olivier Faugeras, Three-Dimensional Computer Vision, Mit press, 1993.



Stereo correspondence (disparity)


Camera Arrays

The Stanford Multi-Camera Array

http://graphics.stanford.edu/projects/array/

to solve:
partly occluding
range data
very high speed video camera
Very high resolution image data