Important: Use custom search function to get better results from our thousands of pages

Use " " for compulsory search eg:"electronics seminar" , use -" " for filter something eg: "electronics seminar" -"/tag/" (used for exclude results from tag pages)
Tags: Distributed, Image, Processing, using, hadoop, MapReduce, framework, mapreduce c, mapreduce combiner, mapreduce cassandra, mapreduce cuda, mapreduce code, mapreduce cloud, mapreduce couchdb, mapreduce criticism, mapreduce counters, mapreduce clustering, mapreduce database, mapreduce diagram, mapreduce drawbacks, mapreduce dean, mapreduce data locality, mapreduce disadvantages, mapreduce debug, mapreduce distributed computing for machine learning, mapreduce design pattern, mapreduce design, mapreduce examples,
Ask More Info Of  A Seminar Ask More Info Of A Project Post Reply  Follow us on Twitter
01-05-2010, 08:13 PM
Post: #1
Distributed Image Processing using hadoop MapReduce framework

.pdf  Distributed Image Processing using hadoop MapReduce framework.pdf (Size: 412.16 KB / Downloads: 726)


Distributed Image Processing using Hadoop MapReduce framework

Presented By:
Binoy A Fernandez
Sameer Kumar
Distributed Image Processing
Objective

To demonstrate how the hadoop mapreduce framework can be extended to work with image data for distributed image processing.
Distributed Sobel Edge Detection algorithm
Easy to understand and implement, and at the same time, it is computationally expensive.
Has the property that the new value to be calculated for a pixel depends not only on that pixel's original value but also on the values of surrounding pixels.
Distributed Image Processing
Basic Idea
An image can be considered as made up of a two dimensional array of pixels. We can partition the image data into subsets and operate on the data in parallel by distributing subsets to different map tasks.
Convolution at a single pixel
1. Create kernel h indexed from 0 to m-1 horizontally and 0 to n-1 vertically and populate it with kernel coefficients
2. Compute kernel half width, m2 = floor(m/2)
Compute kernel half height, n2 = floor(n/2)
3. sum = 0
for k = -n2 to n2 loop
for j = -m2 to m2 loop
sum = sum + h(j + m2, k + n2) f(x - j, y - k)
end loop end loop
g(x, y) = sum
Distributed Image Processing
Edge Detection Algorithm
Convolution of an image ignoring the borders
1. Create kernel h indexed from 0 to m-1 horizontally and 0 to n-1 vertically and populate it with kernel coefficients
2. Compute kernel half width, m2 = floor(m/2)
Compute kernel half height, n2 = floor(n/2)
3. Create an M x N output image, g
4. for all pixel co-ordinates, x and y, loop
g(x, y) = 0
end loop
5. for y = n2 to N-n2-1 loop
for x = m2 to M-m2-1 loop
Compute g(x, y) using previous algorithm end loop end loop
InputFormat
Responsible for,
1. creating the input splits (getSplits)
2. dividing the splits into records (getRecordReader)
InputSplit
A chunk of the input that is processed by a single map.
RecordReader
An input split is divided into records, and the map processes each record (key-value pair) in turn. A RecordReader is used to generate record key-value pairs.
FilelnputFormat
Base class for all implementations of InputFormat that use files as their data source. Provides,
methods to define which files are included as the input.
an implementation for generating splits for the input files.
lnputFormat<K,V> FilelnputFormat<K,V>
TextlnputFormat ImagelnputFormat
Key: LongWntable Key: Text
Value: Text Value: ImageWritable
ImagelnputFormat
Reuse the methods provided for defining which files are to be included as input to a job.
Define a filter to accept only certain image formats.
Define a custom implementation for generating splits for the input files.
Splitting Logic
In putSplit 2
Split Size = 9
(0,9)
[0,13) [0.27) [0,35)
Assumption: Size of an ImageSplit will not be greater than HDFS size.
In putSplit 3
In putSplit 4
Width = 20, Height = 35
Consider image of dimensions 20 x 35 pixels. Based on numSplits argument, the image is divided horizontally
into splits.
Performance: To compute the image splits, we need only the image dimensions. This information can be obtained from the image header thereby avoiding having to read the entire image data into memory.
ImageReader
A record reader implementation for reading sub images based on the computed splits, which is processed independently by the map tasks.
Key: input image path
Value: an ImageWritable
Options :
Read the entire image into memory as a BufferedImage and use the getSubImage method.
Read only the required portion of the image as defined by the split, into memory.
1. Compute the input splits
2. Schedule the map tasks
Submit Job
JobTracker
3- Obtain RecordReader for the split
TaskTracker
Child JVM Map / Reduce
TaskTracker Node
ImageWritable
Custom hadoop type which implements hadoop's serialization framework - Writable interface.
Data to serialize/de-serialize
Size of the image (in bytes)
Start Y co-ordinate relative to input image
Height of the image (in pixels)
Image path
Image data (byte array)
Serialization
How to serialize a Buffered I mage using Hadoop's serialization framework?
1. Serialize RGB colour data
Drawback: Color conversion takes place if the
default model does not match the image
ColorModel.
2. Serialize raw pixel data as a byte array
Drawback: Requires conditional code based on
image type.
3. Serialize in a preferred image format using ImageIO
Advantages:
1) do not have to deal directly with underlying pixel data representation.
2) intermediate image formats can be specified using custom properties.
3) default PNG format used provides lossless zip compression.
OutputFormat
Responsible for,
1. writing output for the job to the file system
(getRecordWriter)
2. checking for validity of output specification of job
(checkOutputSpecs)
RecordWriter
A RecordWriter is used to write the output key-value pairs to an output file.
FileOutputFormat
Base class for all implementations of OutputFormat that use files as their data sink. Provides,
methods to define the output specifications for the job.
an implementation of checkOutputSpecs.
OutputFormat<K,V> FileQutputFormat<K,V>
/ I \
TextOutputFormat MultipleQutputFormat ImageOutputFormat
ImageOutputFormat
ImageOutputFormat is designed similar to the MultipleOutputFormat abstract class, which allows you to
write data to multiple files whose names are derived from
the output keys and values.
ImageOutputFormat provides a RecordWriter implementation for writing the image to the file system.
Key: image path, based on which the output file name is
generated.
Value: image and its metadata, written to file system using ImageIO in the preferred output format.
Map and Reduce function
The map function is an implementation of the Sobel edge detection algorithm applied to the input sub image.
The reduce function uses the metadata associated with each image split to combine them into the final processed output
image.
Key: Image path Value: A sub image
Key: Image path
Value: Processed sub image
Map function
Reduce function!
Key: Image path Value: List of processed &ub images
Key: Image path Value: Final image
Distributed Image Processing
Performance
Cluster Details
Master node
Intel Pentium 4 CPU @ 2.60 GHz, 1.00 GB RAM
Slave node
Intel Core 2 Duo T9550 @ 2.66 GHz. 1.95 GB RAM
Input Data Set
6 hi-resolution PNG images
Largest: 3000 pixels, Smallest: 1080 pixels (height)
Cluster Settings
Maximum concurrent map tasks per tasktracker = 2
Number of reduce tasks for job = 2
Number of splits per image = 2
DFS replication = 2
Distributed Image Processing Performance
Wall Clock Time
Serial Execution (Master) Begin: 8:38:20, End: 8:43:05 Time: 4 minutes, 45 seconds
Parallel Execution (Cluster) Begin: 7:38:05, End: 7:40:55 Time: 2 minutes, 50 seconds
Distributed Image Processing
Enhancements
Handle global and local boundary conditions
Handle images larger than HDFS block size
Evaluate performance of multi-threaded map runner



Please Use Search http://seminarprojects.com/search.php wisely To Get More Information About A Seminar Or Project Topic
15-07-2010, 02:29 PM
Post: #2
RE: Distributed Image Processing using hadoop MapReduce framework
Thanks for uploading. I will learn about it .

Please Use Search http://seminarprojects.com/search.php wisely To Get More Information About A Seminar Or Project Topic
28-09-2010, 05:56 PM
Post: #3
RE: Distributed Image Processing using hadoop MapReduce framework
Interesting topic. I would be nice to discuss more deeply how the image is split. Because as we all know during convolution edge pixels are not computed, so the split images have to overlap with each other.

P.S. are the source code available online?
13-07-2011, 12:07 PM
Post: #4
RE: Distributed Image Processing using hadoop MapReduce framework
Can tell me wat r the hardware and software requirements for the project..?
I have no idea wat a hadoop is?I want to implement this project.Can u Pls guide me.
24-08-2011, 07:16 PM
Post: #5
RE: Distributed Image Processing using hadoop MapReduce framework
Do you provide the source code for this process?
I also want to do some application of image processing based on mapreduce, but I'm a beginner of this framework.So it is difficult for me to write the code for input and output of the image based on the mapreduce framework.Could somebody help me? Thank you in advance.
Rating Distributed Image Processing using hadoop MapReduce framework Options
Share Distributed Image Processing using hadoop MapReduce framework To Your Friends :- Seminar Topics Bookmark
Post Reply 

Marked Categories : image map reduce, ppt on distributed image processing on hadoop, ppt on distributed image processing in hadoop, code for image processing on hadoop, distributed image processing using hadoop mapreduce framework, mapreduce processing images, image processing mapreduce, mapreduce criticism, image processing using mapreduce, hadoop parallel image processing interface ppt, distributed image proceesing in hadoop, meaning of distributive image processing, mapreduce images, hadoop, distributed image processing, mapreduce image processing, hadoop image processing, image processing using hadoop, map reduce code for image processing, hadoop mapreduce splitting the input,

[-]
Quick Reply
Message
Type your reply to this message here.


Image Verification
Image Verification
(case insensitive)
Please enter the text within the image on the left in to the text box below. This process is used to prevent automated posts.

Possibly Related Threads...
Thread: Author Replies: Views: Last Post
  DISTRIBUTED CONTROL SYSTEMS SEMINAR REPORT project maker 0 30 18-09-2014 11:59 AM
Last Post: project maker
  image compression Computer Science Clay 9 3,552 17-09-2014 04:26 PM
Last Post: subine
  Implementation of High-Performance Image Scaling Processor using VLSI Seminar Report seminar code 0 74 04-09-2014 04:32 PM
Last Post: seminar code
  Speech Processing seminar report project maker 0 44 01-09-2014 02:36 PM
Last Post: project maker
  SEMINAR REPORT ON POWER REDUCTION TECHNIQUES IN CMOS IMAGE SENSOR seminar code 0 65 18-08-2014 03:30 PM
Last Post: seminar code
  DISTRIBUTED COMPUTING project maker 0 42 09-08-2014 04:26 PM
Last Post: project maker
  An RTM based Distributed Simulation System for Guide Robot project maker 0 46 01-08-2014 03:01 PM
Last Post: project maker
  IMAGE DEBLURRING FOR RETRIVEL OF CLEAR IMAGES seminar code 0 72 25-07-2014 04:17 PM
Last Post: seminar code
  DIGITAL SIGNAL PROCESSING – An Introduction seminar code 0 83 17-07-2014 02:33 PM
Last Post: seminar code
  1 Mobile, Distributed, and Pervasive Computing project maker 0 58 09-07-2014 10:42 AM
Last Post: project maker
This Page May Contain What is Distributed Image Processing using hadoop MapReduce framework And Latest Information/News About Distributed Image Processing using hadoop MapReduce framework,If Not ...Use Search to get more info about Distributed Image Processing using hadoop MapReduce framework Or Ask Here

Options: