Frame Rate and XML file Test

Test: 1finger.avi
Video Time : 14 sec video
No of Frames : 99

Now if it is true that the video is having 30 fps (frames per sec) when converted to uncompressed AVI then:
No of frames should be = 30 fps * 14 sec = 420 frames but actually it is 99 frames stacked into the program stack.
I think the frames per second is not 30 or may be I am missing something.

Correct me if I am wrong in calculation. We need to figure it out before we give the XML file to signal team and this would help Fan "what to put in time attribute in each event element". Right now she is printing frame no.
here 1 means frame no 1 which does not show the right meaning.

We need to come up with some calculations so that number of frames stacked into the program and fps make sense to produce some values to put in time attribute in each event element of any xml file.

I wont worry about picking up time from video. That is a long term goal and Fan is putting the the time when video is processed and some what make sense. But if we can not relate No of frames produced with video length then the signal team wont be able to analyze our XML file to produce anything meaningful and findings would be meaning less.

Lets work on it. Looking forward to the "video sub team(Peter/Fan)"

-------->
The tested frame was actually 7fps and so 7 X 14 =98 was OK.

General GUI look change

Finger Map Image was opening in a separate window. And I now have idea like this.

How about opening the Finger Map Window in main window (Finger Tracking) rather than in a separate window.

And making the Finger map window smaller (may be by resizing the Finger ID image).

I have an idea like

When the main window opens up –
One the left side put the Finger ID image in one panel and Some Quick start Messages on the right panel(may be console panel is ok with initial quick start messages on it)

When the user clicks on View menu they can make either of these two panels in visible.

Message Panel (or the console) might start with —

1. Choose Video to track Finger from "File" —> Open
2. Select Finger From Combo
3. Click on the particular Finger tip to track from Image
4. If you are advanced user "Set Threshold" otherwise keep the default settings for threshold
5. Hit "Go" for start finger tracking
6. Click "Play" to view tracked finger centroids of your video.

New GUI

Our previous GUI was having buttons and some settings in the mainWindow. I was thinking adding menus as we increase some features. I added some and Kyle did some. I proposed that we have a GUI with finger Map on the left and Console on right with some quick start messages. Users can toggle both Console and Finger Map child windows to view or not. We finally implemented the GUI looks like this.






I liked the check mark before the menu items under "View" menu and those indicates whether the option is checked or not. Looks professional :)

Test with new Multi Finger Tracking

We tested a bigger video file about 2.30 minutes with the new program me and kyle wrote. It could successfully run. :)

However we figured out that this video is not a regular type video. In this video the user did not put their fingers immediately rather delayed couple of seconds before starting exploring the map.

This led us the question :

1)What if the user make delay to explore the map?

Ans: This was a problem because we were loading the first frame to track to give the user an opportunity to select finger tip. Now we decided we can put a slider and load all frames. User would slide to a frame where the finger is visible and click to the finger tip. However we might not need all frames to load at the beginning , I think couple of frames should be enough. This can save time.

Again sometimes users were removing hands from maps so frames with no finger at all are possible.

This led us the question :

2)What if finger moves out of the window, and then comes in at a completely different area?
Ans: We can just put the center as a negative value say , (-1,-1) in our xml file indicating no finger were visible to track in this frame.

We were tracking finger tips in a particular rectangle thinking finger movement won't be that fast and saving search time. But there might be situations user moved out hands to a completely different area or out side the map or moved very fast exceeding the considered search space.

This lead to question:

3) If center is not found the next search space in the frame, then how to search?

Ans: Simple way is to search every pixels in the frame considering the whole pixel grid rather a the previously considered small space. This should work if finger is moved out fast or to a different area. But if finger is outside the frame then put some -ve values as centroid say (-1,-1)

Peer Programing - Test of Multi Finger Tracking with new Algorithm

Me an Kyle sat together to test my proposed algorithm for multi finger tracking. We changed the complete old code where user needed to say which color to track and which color to compare with, what is the value of the tracking color by which it is greater in some extent than other comparing colors. These were hassle and ambiguous for a new user and a user who is also familiar need to remember these sort of values or settings which is a pain. Instead my proposal is that if somehow user can point the exact finger tip s/he is interested with and if we can get the exact pixel color then we can track the exact color. This is simple and very much user friendly.

It worked after we spent around 3 hours. We could trase a finger with my proposed way. Algorithm was simple enough, we just needed to use some thresholding. It worked because we were tracking the exact color of the finger color rather than assuming "red", "blue", "green" combination of the finger.

Multi Finger Tracking with advanced UI

We would be having Menu and different settings in our next version of UI. When the user opens a video the frame should come up and user would select fingers from frame and have the finger ID checked in the corresponding checkBox.

Something to figure out :

1. What if the first frame does not come with fingers on it?
Probable Solution: Let the user slide until there is a frame with finger colored. How many frames would come up then before actual tracking begin? We can load all frames before actual tracking if needed. Best thing would be input a video which start color fingers on it to avoid necessary delay.

2. Let the user choose one finger at a time because finger ID need to match with the finger tip a user selects.

3. Threshold checking to match color pixels to see how pixel's color fluctuates due to shade or anything else. We can also make this feature as an advanced settings for advanced user rather users should use default threshold that our program sets.





There should be play, pause, stop, sliding bar and a progress bar in the second window that comes up when a user feeds a video into the program.

I have an idea of having the UI design done soon.

Features for the Next Version

Needs:

• Multi - Finger Tracking

• Good UI -> Settings | Help

• Finalized XML

• Good Templated Docs

• Stress Test/ Nagative Value Test

• Coordinate Conversion/Scaling

Easy:

• Progress Bar

• Help Menu

• Settings

• Batch Processing

Hard:

• Video conversion

• Image Registration