Course pages 2016–17
Lectures for this course are delivered in the Engineering Department:
Practical exercises will be introduced and assessed by teaching and research staff in the Computer Laboratory. The total amount of self-study time devoted to the course should be approximately 64 hours. The first two practical exercises should be completed in about 8-12 hours (exercises 1 and exercise 2) and the mini-project in about 40-50 hours.
Step 1: Install and set up OpenCV (see notes below). Read some tutorials to gain basic familiarity with OpenCV and its core methods and capabilities. (about 1.5 hours)
Step 2: Use OpenCV to carry out camera calibration following the steps described in http://docs.opencv.org/2.4/doc/tutorials/calib3d/camera_calibration/camera_calibration.html (about 1.5 hours)
Step 3: Take a set of photos of the rubber stamp impressions in your ACS Research Skills logbook. Write code that makes use of OpenCV to load each image file in turn and detect the outer corners of the stamp. Then use the co-ordinates of those corners to produce (to a good approximation) a perspective corrected output image for each input image, i.e. the output should show a "top-down" view of stamp without major distortions (any "in-plane" rotation of the stamp image around the axis normal to the image plane may be ignored). (about 3 hours)
Step 4: Write the report. The report should have two sections of no more than two pages each:
a) Camera calibration: list the intrinsic and extrinsic camera parameters that were obtained during the camera calibration. Briefly describe, using the theory covered in the lecture course, the meaning of these parameters and how OpenCV determines their values.
b) Perspective correction: List the main OpenCV methods you employed and briefly describe their purpose and underlying algorithms. Under what conditions are 3, rather than 4 or more, points on the object sufficient to perform a perspective correction? How could knowledge of the intrinsic and extrinsic camera parameters lead to an improved solution?
As an example of writing style that is appropriate for straightforward reporting of a computer vision project, you may like to refer to the following paper by Chris Town:
Step 5: Submit the report as PDF, with an appendix containing your code written in any high level language with OpenCV bindings (C++, Python, Java or Matlab).
1. Obtain data: [.5 h]
Download the provided sub-set of "Places" dataset described by Zhou et al. This includes a training set, testing set and class labels.
Temporarily hosted (this link will soon change) at: https://www.dropbox.com/sh/riw3s6nc8z7l6pw/AACnCnOEM1XGkMQKyDoRGTz1a?dl=0
- B. Zhou, A. Lapedriza, J. Xiao, A. Torralba, and A. Oliva. Learning Deep Features for Scene Recognition using Places Database. Advances in Neural Information Processing Systems 27 (NIPS), 2014.
2. Feature extraction: [3h]
Choose a set of reasonable features to extract. These can include:
- Appearance features
- Image filters
- Local features
- Colour features
At least two different types of features need to be used and extracted from each of the images. Explain your choice of features. Include sample images for each feature extraction stage in your report.
3. Classification: [3h]
Using the training set, train a linear SVM using your features and calculate classification accuracy using different feature sets as follows:
- Using single features separately.
- Using a multimodal / combination of features.
LibSVM is an open source library for SVM implementation: http://www.csie.ntu.edu.tw/~cjlin/libsvm/
There is also an openCV tutorial for SVMs: http://docs.opencv.org/2.4/doc/tutorials/ml/introduction_to_svm/introduction_to_svm.html
4. Report: [2h]
In your report, describe your selection of features, training and classification results. Discuss classification results: why do you think some features had better or worse performance ?
5. Possible extensions:
- Compare with more complex machine learning - e.g. Complex kernel SVMs
- Evaluate the generalisability of your approach on images outside the provided dataset. Take a set of photos of different scenes (within the provided image classes) and use your previously trained model to classify them. (you can use photos from the internet as well)
Please prepare a one-page proposal, addressing the following questions:
- Your personal starting point - what have you done in this area before?
- General approach: will your mini-project focus on mathematical theory, novel algorithms or engineering applications?
- What is the original research question or goal of your mini-project??
- Summarise the technique you plan to use, or theory you plan to apply, with reference / citations/
- Explain what materials you need access to, such as data sets, cameras, specialised libraries etc
- Estimate the computational resources you will need for the project, including memory, disk, CPU hours
Exercises will be based on OpenCV (Open Source Computer Vision Library: http://opencv.org.
Open CV is an open-source BSD-licensed library that includes some hundreds of computer vision algorithms. OpenCV has C++, C, Python and Java interfaces and supports Windows, Linux, Mac OS, iOS and Android. You should choose the language and environment to use for your work on this course, taking into account your own experience and resources. Please consult one of the teaching staff is you are unsure about this choice.
Information on installation and setup of OpenCV for a variety of platforms is available at: http://docs.opencv.org/2.4/doc/tutorials/introduction/table_of_content_introduction/table_of_content_introduction.html
Information on the core functionality of OpenCV is available at: http://docs.opencv.org/2.4/doc/tutorials/core/table_of_content_core/table_of_content_core.html#table-of-content-core
Other resources include:
- Robert Laganière, OpenCV 2 Computer Vision Application Programming Cookbook, PACKT publishing. (available online)
- Daniel Lélis Baggio et al. Mastering OpenCV with practical computer vision projects. Packt Publishing Ltd.