(symbol processing) (pattern recognition) (content) (identify) (interpret) (raw data) - 1 -
9D-SPA - 2 -
2D string (Chang, Shi, and Yan, 1987) 2D G-string (Jungert, 1988) 2D C-string (Lee and Hsu, 1990) 2D C + -string (Huang and Jean, 1994) 2D Z-string (Lee and Chiu, 2003) unique-id-based matrix (Chang, Ann, and Yeh, 2000) prime number based matrix (Chang, Yang, and Yeh, 2001) 9DLT matrix (Chang, 1991) 9D-SPA (Huang and Lee, 2004) RS-string (Huang and Jean, 1996) OG-string (Jean and Chang, 2006) 2D string Chang (1987) 2D string < = x y < = xy 2-1(a) 2D string - 3 -
x A=B<D<C y D<B=C<A u-string v-string A B C A B C D D (a) (b) 2-1 2D G-string 2D string 2-1(b) 2D string 2D G-string (Jungert, 1988) 2D string 2-1(b) 2D G-string 2D G-string x A A=B A=B=D A=D D C=D y D<B B=C A=B=C A=C A 2D C-string 2D C + -string - 4 -
2D G-string 2D C-string (Lee and Hsu, 1992) 2D G-string 2D C-string 7 13 2-1 2-1(b) 2D C-string x A](B]D D) D]C y D<B](C]A) (A[C) 2-1 2D C-string A < B End(A) < Begin(B) A B A B End(A) = Begin(B) A B A % B Begin(A) < Begin(B), End(A) > End(B) A B A [ B Begin(A) = Begin(B), End(A) > End(B) A B A ] B Begin(A) < Begin(B), End(A) = End(B) A B A / B Begin(A) < Begin(B) > End(A) > End(B) A B A = B Begin(A) = Begin(B), End(A) = End(B) A B A <* B End(B) < Begin(A) B A A * B End(B) = Begin(A) B A A %* B Begin(B) < Begin(A), End(B) > End(A) B A A [* B Begin(B) = Begin(A), End(B) > End(A) B A A ]* B Begin(B) < Begin(A), End(B) = End(A) B A A /* B Begin(B) < Begin(A) > End(B) > End(A) B A S. Y. Lee and F. J. Hsu (1992). Spatial reasoning and similarity retrieval of images using 2D C-string knowledge representation. Pattern Recogn, 25(3), 305-318. 2D C-string - 5 -
Huang and Jean (1994) 2D C + -string x y 2-1(b) 2D C + -string 2D C + -string 2D C-string x A 5 ](B 2 ]D 1 D 1 ) D 6 ]C 2 y D 2 < 1 B 5 ](C 4 ]A 1 ) (A 2 [C 1 ) 2D Z-string 2D Z-string (Lee and Chiu, 2003) 2D C + -string (1) A s A x y s (2) A< d B A B (Begin(B) - End(A)) d (3) A% d B A B (Begin(B) - Begin(A)) d (4) A/ d B A B A B (End(A) - Begin(B)) d 2-1(b) 2D Z-string x (A 5 % 3 B 2 )/ 2 (D 8 ]C 2 ) y (D 2 < 1 ((B 5 / 4 C 5 )/ 2 A 3 )) 9DLT matrix 9DLT matrix (nine direction lower-triangular matrix) (Chang, 1991) 9 9 2-2 - 6 -
2-2 R 0 R 1 R 2 R 3 R 4 R 2-1(a) 2-1(b) 2-3 1 2 8 3 R 0 7 4 6 5 2-2 9DLT C. C. Chang (1991). Spatial match retrieval of symbolic pictures. Journal of Information Science and Engineering, 7(3). 405-422. A B C D A B C D A - - - - A - - - - T(a)= B 5 - - - T(b)= B 5 - - - C 6 7 - - C 6 7 - - D 6 6 4 - D 6 6 4-2-3 9DLT matrix 2-2 - 7 -
D pq D qp T pq O pq D pq(d qp) O pq i 9D-SPA 9D-SPA (Huang and Lee, 2004) 9DLT 9D-SPA R={(O pq D pq D qp T pq ) (O pq D pq D qp T pq )} ( p q i j ) p q O pq O pq = (q-1)(q-2)/2 + p 1 p<q n n j 9 8 1 0 9 2-4 j i D pq D qp T pq 4 0= (disjoin) 1= (join/meet) 2= (partly_overlap) 3= (contain/inside) 2-1(b) 9D-SPA R = {(1,31,64,2),(2,24,129,0),(3,48,3,0),(4,12,192,0), (5,12,192,0),(6,4,96,0)} Area 4 (00001000) 2 = 8 Area 5 (00010000) 2 = 16 Area 6 (00100000) 2 = 32 Area 3 (00000100) 2 = 4 Area 0(MBR) (00000000) 2 = 0 Area 7 (01000000) 2 = 64 Area 2 (00000010) 2 = 2 Area 1 (00000001) 2 = 1 Area 8 (10000000) 2 = 128 2-4 9D-SPA P. W. Huang and C. H. Lee (2004). Image database design based on 9D-SPA representation for spatial relation. IEEE Transactions on Knowledge and Data Engineering, 16(12), 1486-1496. - 8 -
2D C-tree (Hsu, Lee, and Lin, 1999) 9DLT string (Chan, and Chang, 2001) 3D C-String (Lee, Chiu, and Yu, 2002) 3D Z-string (Lee, Yu, Chiu, and Hong, 2005) AVIS (advanced video information system) (Koprulu, Cicekli, and Yazici, 2003) FPI-tree AFPI-tree (Su, Huang, Yeh, and Tseng, 2010) motion-based scene tree (Yi, Rajan, and Chia, 2005) 2D C-Tree 2D C-Tree (Lee, Hsu, and Lin, 1998) 2D string x y R 2D C-Tree 2D C-Tree (Hsu et al., 1999) 2D C-Tree x y 2D C-Tree 2D C-Tree 2D C-Tree - 9 -
full-sequence matching segment matching subsequence matching 2-5 Video I 2D C-Tree 2-6 2-7( ) x y C A A A B B B Frame 1 Frame 2 Frame 3 C C D D D A A A B B B Frame 4 Frame 5 Frame 6 2-5 Video I R X F 1 F 2 F 3 F 4 F 5 A ε B A ε B C A 2 ε B C ε B ε D C B ε D ε A 1 ε A ε A ε 2-6 2D C-Tree Video I 1 5-10 -
R Y F 1 F 2 F 3 F 4 F 5 A A A ε C D D B ε B ε B ε ε A ε C B A 2 ε C B ε ε A 1 2-7 2D C-Tree Video I 1 5 AVIS AVIS (advanced video information system) Adali, Candan, Chen, Subrahmanian, and Erol (1996) AVIS AVIS (Koprulu et al., 2003) AVIS ( ) (a) (b) [R S,R E ) R S R E - 11 -
R E (c) (d) 2-5 Video I 2-8 Video I B B Girl B 5 6( ) 5 B (x,y) (4,2) (9,4) 6 B (4,0) (6,3) 2-9 Video I B 2-10 A A Boy B Girl C Bird 7 [(1,2),(3,5)] 5 [(4,2),(9,4)] 2 [(0,7),(3,9)] 6 [(4,0),(6,3)] 3 [(0,7),(4,9)] D Tree 6 [(8,1),(10,9)] 2-8 AVIS Video I D Object C B A 0 1 2 3 4 5 6 Frame 2-9 AVIS Video I - 12 -
7 [1,6) A 5 [1,4) B 6 [4,6) BD 1 [1,3) 2 [3,4) C 3 [4,5) C 4 [5,6) 2-10 AVIS Video I 3D Z-string 3D Z-string (Lee et al., 2005) 2D Z-string 3D Z-string u-string v-string t-string u-string x v-string y t-string t 2D Z-string # t-string A A 8#2#6 A 8 2 A 3 8 v,r v,r v r 2-5 - 13 -
Video I 2-11 Video I 3D Z-string 3D Z-string u-string ((C 2 0.5,2 / 1 A 2 0,1 )< 4 (B 2,1 0,1 / 1 D 2 0,1 )) v-string (D 8 0,1 ](A 3 4,1 [B 2 0,1 1,1 )< 3C 2 0,1 )) t-string ((A 6 =B 6#4#2 )](C 3#3 / 1 D 3 )) C A B D 2-11 Video I FPI-tree AFPI-tree FPI-tree (fast pattern index tree) pattern index tree) Su (2010) AFPI-tree (advanced fast 2-2 A G Dwinsize Dwinsize Dwinsize n n ( ) Two shot-pattern Two shot-pattern FPI-tree - 14 -
Two shot-pattern AFPI-tree 2-2 Clip 2 2-3 2-4 Dwinsize 3 Dwinsize 4 Two shot-pattern 2-2 FPI-tree Clip ID Key-Frame Clip 1 A, B, C, A Clip 2 C, B, B, A, E, F Clip 3 F, F, E, E, A, B, D, B, C, A, B Clip 4 B, C, G, C, A, D, B 2-3 Two shot-pattern Clip 2 Dwinsize=3 Clip 2: C, B, B, A, E, F Starting Shot Two shot-pattern (FPI) Two shot-pattern (AFPI) C C B,C A C B,C B,C A B B B,B A,B E B B,B A,B E B B F B A,B E,B F A A E,A F A E,A F E E F E F 2-4 Two shot-pattern Clip 2 Dwinsize=4 Clip 2: C, B, B, A, E, F Starting Shot Two shot-pattern (FPI) Two shot-pattern (AFPI) C C B,C A,C E C B,C B,C A,C E B B B,B A,B E,B F B B,B A,B E,B F B B A,B E,B F A A E,A F A E,A F E E F E F shot-pattern Two shot-pattern (parent node) Two shot-pattern AFPI-tree Two Two - 15 -
shot-pattern Two shot-pattern 2-12 2-13 2-2 FPI-tree AFPI-tree Dwinsize 3 A B C D E F G 1 134 1 34 2 2 A B C D E F G 123 A B C D E F G 1234 A B C D E 23 234 34 134 3 2 2 4 4 4 4 F G A B C D E 3 3 3 F G 2-12 FPI-tree 2-2 3 3 3 2 A B C D E F G 3 3 3 A B C D E F G 4 4 4 A B C D E F G A B C D E F G 1,1/6 1,1/6 3,3/27 4,1/15 1,1/6 3,1/27 4,1/15 2,1/12 2,1/12 A B C D E F G 1,1/6 2,2/12 3,1/27 2,1/12 3,2/27 1,1/6 3,2/27 4,2/15 3,1/27 2,2/12 2,2/12 4,1/15 A B C D E F G 1,1/6 2,1/12 3,1/27 4,2/15 2,2/12 3,1/27 4,1/15 4,1/15 4,1/15 4,1/15 A B C D E F G 3,1/27 3,1/27 A B 3,1/27 3,2/27 4,1/15 C D E F G 3,2/27 3,1/27 3,1/27 2,1/12 A B C D E F G 2-13 AFPI-tree 2-2 3,1/27 3,4/27 3.1/27 A B C D E F G 4,1/15 4,1/15 4,1/15 A B C D E F G - 16 -
2-12 (A B) Two shot-pattern Clip 1 Clip 3 Clip 4 2-13 (A B) 1/6 3/27 1/15 AVIS 2-5 AFPI-tree 2D C-tree 3D Z-string 3D Z-string 9D-SPA 9D-SPA 9D-SPA - 17 -
9D-SPA 9D-SPA 9D-SPA 2-5 2D C-tree AVIS 3D Z-string AFPI-tree O X O X O O O O O X O X ( ) O X - 18 -
9DLT 9DLT 9DLT 2D C-trees 3D Z-string 9D-SPA bounding rectangle) (minimum - 19 -
x y [(F,X d,y d,x z,y z ) 1 (F,X d,y d,x z,y z ) n ] [ObjectID][t,c,x,y,w,h] 6 (t) (c) x (x) y (y) (w) (h) 1 (F) X d (Y d ) X z (Y z ) x y x (y ) 2-5 6 Video I A B C D 1 2 3 4 Video I: - 20 -
[1][1, 6, 2, 3.5, 2, 3] [2][1, 6, 8, 3, 2, 2][4,-1, 0, 0, 0 2,0, -1, 0, 0] [3][3, 3, 1, 8, 2, 2][3,0.5, 0, 1, 0] [4][4, 3, 9, 5, 2, 8] 2-5 Video I A [1,6,2,3.5,2,3] A 1 6 (2,3.5) 2 3 A B [1,6,8,3,2,2] B 1 6 (8,3) 2 2 [4,-1,0,0,0 2,0,-1,0,0] B 4 2 [X d,y d,x z,y z ] [-1,0,0,0] [0,-1,0,0] B -1 B -1 C [3,3,1,8,2,2] C 3 3 (1,8) 2 2 [3,0.5,0,1,0] 3 [0.5,0,1,0] C 0.5 1 D - 21 -
Y T 3-1 3-1 OID, X L, X R, Y D, Y T ] F N [F N, OID X L X R Y D Video I OID F N, OID, X L, X R, Y D, Y T ObjectID X Y W H T X=(X L +X R )/2 W=X R -X L Y=(Y D +Y T )/2 H=Y T -Y D - 22 -
Video I Video I Video I 1 2 3 4 1 4 B( 2) X L X R Y D Y T [7, 9, 2, 4] X Y W H [8, 3, 2, 2] F S [6, 8, 2, 4] [7, 3, 2, 2] V X V Y V W V H [-1, 0, 0, 0] [5, 7, 2, 4] [6, 3, 2, 2] [-1, 0, 0, 0] [4, 6, 2, 4] [5, 3, 2, 2] [-1, 0, 0, 0] [4, 6, 1, 3] [5, 2, 2, 2] [0, -1, 0, 0] F S F N 1 F E F N 4 C T F E -F N +1 4 C T V X V Y V W V H [4,-1,0,0,0] F N F S [4, 6, 0, 2] [5, 1, 2, 2] [0, -1, 0, 0] F N F E C T F E -F N +1 2 [2,0,-1,0,0] A C D - 23 -
1, 1, 1, 3, 2, 5 1, 2, 7, 9, 2, 4 2, 1, 1, 3, 2, 5 2, 2, 6, 8, 2, 4 3, 1, 1, 3, 2, 5 3, 2, 5, 7, 2, 4 3, 3, 0, 2, 7, 9 4, 1, 1, 3, 2, 5 4, 2, 4, 6, 2, 4 4, 3, 0, 3, 7, 9 4, 4, 8,10, 1, 9 5, 1, 1, 3, 2, 5 5, 2, 4, 6, 1, 3 5, 3, 0, 4, 7, 9 5, 4, 8,10, 1, 9 6, 1, 1, 3, 2, 5 6, 2, 4, 6, 0, 2 6, 4, 8,10, 1, 9 1, 1, 1, 3, 2, 5 2, 1, 1, 3, 2, 5 3, 1, 1, 3, 2, 5 4, 1, 1, 3, 2, 5 5, 1, 1, 3, 2, 5 6, 1, 1, 3, 2, 5 1 3, 3, 0, 2, 7, 9 4, 3, 0, 3, 7, 9 5, 3, 0, 4, 7, 9 3 1, 2, 7, 9, 2, 4 2, 2, 6, 8, 2, 4 3, 2, 5, 7, 2, 4 4, 2, 4, 6, 2, 4 5, 2, 4, 6, 1, 3 6, 2, 4, 6, 0, 2 X Y W H 2 4, 4, 8,10, 1, 9 5, 4, 8,10, 1, 9 6, 4, 8,10, 1, 9 4 X = (X L +X R )/2 Y = (Y D +Y T )/2 W = X R - X L H = Y T - Y D P S C T V X V Y V W V H P S C T V X V Y V W V H 1 6 0 0 0 0 P S C T V X V Y V W V H 1 4-1 0 0 0 5 2 0-1 0 0 1 2 P S C T V X V Y V W V H 3 3 0.5 0 1 0 P S C T V X V Y V W V H 4 3 0 0 0 0 3 4 [1][1,6,2,3.5,2,3]; [2][1,6,8,3,2,2][4,-1,0,0,0 2,0,-1,0,0]; [3][3,3,1,8,2,2][3,0.5,0,1,0]; [4][4,3,9,5,2,8]; 3-1 Video I - 24 -
3-1 Function ( : ) : (OITable), (T X,T Y,T W,T H ), (TV X,TV Y,TV W,TV H ), (F S ), (F E ) ; For Each In For Each In : (OID), (F N ), (X L ), (X R ), (Y D ), (Y T ); x (X) = X L, y (Y) = Y D, (W) = X R - X L, (H) = Y T - Y D ; If F S = F N (T X, T Y, T W, T H ) = (X,Y,W,H) OITable OITable (ObjectID) (1) = OID OITable (t,x,y,w,h) (1) = (F N,X,Y,W,H) Else x (V X ) = X - T X, y (V Y ) = Y T Y, (V W ) = W T W, (V H ) = H T H ; If (TV X, TV Y, TV W, TV H ) = (V X, V Y, V W, V H ) ElseIf F E = F N - 1 (C T ) = F E - F S + 1 OITable (F,X d,y d,x z,y z ) (1) = (C T,TV X,TV Y,TV W,TV H ) (TV X,TV Y,TV W,TV H ) = (V X,V Y,V W,V H ) F S = F N End If End If If OITable (c) (1) = (F N t + 1) (1) F E = F N ( ) - 25 -
3-1( ) (C T ) = F E - F S OITable (F,X d,y d,x z,y z ) (1) = (C T,TV X,TV Y,TV W,TV H ) End If Next Next OITable End Function (1) 2) 3-2 3-2 Video I ( 1 1 x X X V X y Y Y V Y W H X Y W H X L X R Y D Y T X L =X-W/2 X R =X+W/2 Y D =Y-H/2 Y T =Y+H/2-26 -
3-2 Function ( : ) : (OATable) (T X,T Y,T W,T H ) For Each In (OID) (F N ) (T X,T Y,T W,T H ) = (x,y,w,h) (2) (X L ) (X R ) (Y T ) (Y D ) OATable (F N,OID,X L,X R,Y D,Y T ) (1) (F Now ) = F N For Each In (C T,V X,V Y,V W,V H ) For 1 To C T F Now = F Now + 1 (New X,New Y,New W,New H ) New X = T X + V X, New Y = T Y + V Y New W = T W + V W, New H = T H + V H ; (X L ) (X R ) (Y T ) (Y D ) OATable (F Now,OID,X L,X R,Y D,Y T ) (1) (T X,T Y,T W,T H ) = (New X, New Y, New W, New H ) Next Next Next OATable (F Now ) (1) (OID) (1) OATable End Function (1) (2) - 27 -
[1][1,6,2,3.5,2,3][6,0,0,0,0]; [2][1,6,8,3,2,2][4,-1,0,0,0 2,0,-1,0,0]; X L X R Y D Y T 1 2 F S X Y W H 1 2 3.5 2 3 ( ) C T V X V Y V W V H 6 0 0 0 0 1 F S X Y W H 1 8 3 2 2 ( ) C T V X V Y V W V H 4-1 0 0 0 ( ) C T V X V Y V W V H 2 0-1 0 0 2 F N OID X L X R Y D 1 1 1 3 2 1 Y T 5 2 1 1 3 2 5 3 1 1 3 2 5 4 1 1 3 2 5 5 1 1 3 2 5 6 1 1 3 2 5 F N OID X L X R Y D 1 2 7 9 2 2 Y T 4 2 2 6 8 2 4 3 2 5 7 2 4 4 2 4 6 2 4 5 2 4 6 1 3 6 2 4 6 0 2 3-2 Video I A B - 28 -
9D-SPA (Huang and Lee, 2004) 9 D pq D qp 4 T pq S pq p q 13 2D C-string (Lee and Hsu, 1992) 2-1 * p<*q q<p 3-3 VideoID = {[(O pq ) (S pq ) (Relation) (Shot)]} O pq 9D-SPA S pq Shot (F s1 F e1 F sk F ek ) F s F e Shot Relation Relation D pq D qp T pq Shot Shot 3-3 Symbol < / ] % [ = Value 0 1 2 3 4 5 6 Symbol <* * /* ]* %* [* Value 7 8 9 10 11 12 * 3-4 Video I A B C D 1 2 3 4-29 -
3-5 Video I (O pq ) 1 2 3 4 5 6 (1,2) (1,3) (2,3) (1,4) (2,4) (3,4) Video I A 1 B 2 3-4 AB O pq (q-1)(q-2)/2+p (2-1)(2-2)/2+1 1 AC AD BD BC CD 2 3 4 5 6 3-5 1 A B S pq A=B 3-3 6 1~4 D pq D qp T pq (24 1 0) Shot 1 (1-4) Relation 1 (24,1,0) 5 D pq D qp T pq (24 129 0) Shot 2 (5) Relation 2 (24,129,0) 6 D pq D qp T pq (8 128 0) Shot 3 (6) Relation 3 (8,128,0) Video I Video I = { [(1) (6) (24,1,0 24,129,0 8,128,0) (1-4 5 6)]; [(2) (4) (192,12,0 64,12,0 64,14,0) (3 4 5)]; [(3) (4) (128,8,0) (3-5)];[(4) (3) (16,131,0) (4-6)]; [(5) (3) (16,131,0 16,3,0 48,3,0) (4 5 6)]; [(6) (2) (16,129,0) (4-5)];} - 30 -
O q O p (O p O q ) 3-6 3-3 ( 2 ) 3-3 O q (1,2) (1,3) (2,3) (1,4) (2,4) (3,4) 3-6 Function ( : ) : (able) For Each Q In For Each P(OID P <OID Q ) In able PQ (O pq ) PQ M N (S pq ) able (O pq ), able (S pq ); (TD pq,td qp,tt pq ) For Each (F N ) In PQ PQ (D pq,d qp,t pq ) If (D pq,d qp,t pq ) (TD pq,td qp,tt pq ) able (TD pq,td qp,tt pq ) able (F N ) Else able (F N ) End If (TD pq,td qp,tt pq ) = (D pq,d qp,t pq ) Next Next Next able End Function - 31 -
[1][1,6,2,3.5,2,3]; [2][1,6,8,3,2,2][4,-1,0,0,0 2,0,-1,0,0]; [3][3,3,1,8,2,2][3,0.5,0,1,0]; [4][4,3,9,5,2,8]; OID: 1 OID: 2 OID: 3 OID: 4 O 12 : 1 O 13 : 2 O 23 : 3 O 14 : 4 O 24 : 5 O 34 : 6 Object Start Frame End Frame 1 1 1+5=6 Object 1: Object 3: 1 3 5 6 3 3 3+2=5 1 3 1 % 3 S 13 = 4 Fram OID X 1 X 2 Y 1 Y 2 3 1 1 3 2 5 3 0 2 7 9 4 1 1 3 2 5 3 0 3 7 9 5 1 1 3 2 5 3 0 4 7 9 9D-SPA D 13 D 13 T 13 192 12 0 64 12 0 64 14 0 (2) (4) (192,12,0 64,12,0 64,14,0) (3 4 5) Video A 2 3-3 Video I O pq =(q-1)(q-2)/2+p (1,3) 2 S pq (1,3) 1 1 6 3-32 -
3 5 O 1 %O 3 3-3 S pq 4 (1,3) 3 5 (1,3) 3 (D pq, D qp, T pq ) (192,12,0) 4 (64,12,0) 5 (64,14,0) (1,3) O pq 2 S pq 4 3 (192,12,0) 4 (64,12,0) 5 (64,14,0) (1,3) (O pq )(S pq )(Relation)(Shot) (2) (4) (192,12,0 64,12,0 64,14,0) (3 4 5) 3-4 9D-SPA (D) (S) 3 (T) D pq D qp D pq D qp VID Shot D pq D qp 4 T pq 0 3 T pq 13 S pq 0 12 S pq - 33 -
O ij = 1 D D ij D ji T VID:Shot O ij = N S D T S T ij =0 VID:Shot 0 VID D ij 1 VID D ji VID:Shot 3-4 Next D ij D ji VID:Shot T ij =3 12 VID Next D ij D ji VID:Shot T ij =0 T ij =3 VID:Shot 0 VID 1 VID VID:Shot VID:Shot 12 VID Next Next C C A A A B B B Frame 1 Frame 2 Frame 3 C D C D D A A A B B B Frame 4 Frame 5 Frame 6 Video II = { [(1) (6) (24,1,0 24,1,1) (1-5 6)] [(2) (4) (192,12,0 64,4,0 96,6,0 32,2,0) (2 3 4 5)] [(3) (4) (128,8,0 192,12,0) (2-4 5-5)] [(4) (3) (16,131,0) (4-6)] [(5) (3) (16,131,0) (4-6)] [(6) (2) (16,129,0) (4-5)] } 3-5 Video II - 34 -
1 D 8 128 Next 24 1 Next 24 129 End T I:(6, 6) I:(1, 4);II:(1, 6) I:(5, 5) S 0 1 2 3 I:(1, 6);II:(1, 5) II:(6, 6) 0 1 2 3 4 5 6 7 8 9 10 11 12 I;II 3 D T S 5 D T S 128 8 Next 192 12 End I:(3, 5);II:(2, 4) II:(5, 5) 0 1 2 3 I:(3, 5);II:(2, 5) 0 1 2 3 4 5 6 7 8 9 10 11 12 I;II 16 3 Next 16 131 Next 48 3 End I:(5, 5) I:(4, 4);II:(4, 6) I:(6, 6) 0 1 2 3 I:(4, 6);II:(4, 6) 0 1 2 3 4 5 6 7 8 9 10 11 12 I;II 3-6 Video I Video II 2-5 Video I 3-5 Video II 3-6 1 2 O pq 1 (D) (8,128) I 6 (24,1) I 1 4 II 1 6 (24,129) I 5 (T) T pq 0 I 1 6 II 1 5 T pq 1 II 6 (S) 6-35 -
3-3 I II AB 3-7 Function ( : ) : " (S)" For Each (VID) For Each In (O pq ), (S pq ), (Relation) If S S (O pq ) " (DArray)", " (SArray)" 12, " (TArray)"; 4 End IF S(O pq )_SArray (S pq ) (VID) For Each Relation (Shot) (D pq ),(D qp ), (T pq ) S(O pq )_TArray (T pq ) (VID)(Shot) If S(O pq )_DArray (D pq, D qp ) S(O pq )_DArray (D pq, D qp ) (D pq ) (D qp ) End If S(O pq )_DArray (D pq, D qp ) (VID)(Shot) Next Next Next S End Function 3-7 - 36 -
(D pq, D qp ) D pq D qp (D pq, D qp ) T pq 0 4 13 4 S pq 3-5 A B B A A 1 B 2 1 (D pq, D qp ) (24,1) I: (1,4) II: (1,6) Video I 1 4 Video II 1 6 3-7 A B A B 1 A=B 3-3 6 I II - 37 -
3-8 1 D 8 128 Next 24 1 Next T I:(6, 6) I:(1, 4);II:(1, 6) S 3-7 0 I:(1, 6);II:(1, 5) 1 II:(6, 6) 2 3 0 1 2 3 4 5 6 7 8 9 10 11 12 I;II 24 129 End I:(5, 5) (O pq, Type, Value 1, Value 2 ) O pq Type 0 2 Value 1 (Value 2 ) D pq (D qp ) T pq S pq (OR) (AND) 1 D 8 128 Next 24 1 Next T I:(6, 6) I:(1, 4);II:(1, 6) S 3-8 0 I:(1, 6);II:(1, 5) 1 II:(6, 6) 2 3 0 1 2 3 4 5 6 7 8 9 10 11 12 I;II 24 129 End I:(5, 5) - 38 -
3-6 A B B A (1,0,24,1) 1 A B 0 24 1 D pq D qp A B (1,2,6,0) 1 A B 2 6 0 S pq 6 A=B 0-39 -
VB.NET Windows XP Professional 2002 Service Pack 3 CPU Intel Core2 Quad Q8300 2.50GHz RAM 2.50GHz 3.00GB Microsoft Visual Studio 2008 4-2 4-1 Save Directory 4-3 4-1 - 40 -
4-2 4-3 4-4 4-5 4-6 - 41 -
4-4 4-5 - 42 -
4-6 4-7 4-8 4-9 - 43 -
4-7 4-8 - 44 -
4-9 4-10 4-11 - 45 -
4-10 4-11 - 46 -
(V O ) (V F ) (V C ) 100 4-12 V O V F V C 10000 5 4-1 4-1 V O V O ExecutiveTime(s) 100 311.82 200 633.99 300 946.42 400 1263.93 500 1561.76 600 1905.39 700 2220.47 800 2546.79 900 2865.44 1000 3185.66-47 -
Executive Time(second) 3500 3000 2500 2000 1500 1000 500 0 100 200 300 400 500 600 700 800 900 1000 Number of Objects 4-12 V O 4-13 V F V O V C 100 5 4-2 4-2 V F V F ExecutiveTime(s) 10000 311.82 20000 633.82 30000 935.43 40000 1286.45 50000 1625.46 60000 1871.79 70000 2289.42 80000 2571.67 90000 2901.85-48 -
3500 3000 Executive Time(second) 2500 2000 1500 1000 500 0 10K 20K 30K 40K 50K 60K 70K 80K 90K Number of Frames 4-13 V F V C V O V F 100 10000 4-3 4-14 V C 4-3 V C V C ExecutiveTime(s) 1 68.81 2 137.04 3 198.62 4 259.51 5 311.82 6 361.59 7 406.54 8 459.26 9 501.54 10 541.00-49 -
Executive Time(second) 600 500 400 300 200 100 0 1 2 3 4 5 6 7 8 9 10 Max. Number of Changed States 4-14 V C V F V C 10000 5 4-4 4-15 V O 4-4 V O V O ExecutiveTime(s) 100 62.78 200 128.84 300 195.95 400 261.67 500 326.57 600 398.20 700 465.65 800 533.57 900 605.07 1000 676.01-50 -
Executive Time(second) 800 700 600 500 400 300 200 100 0 100 200 300 400 500 600 700 800 900 1000 Number of Objects 4-15 V O V O V C 100 5 4-5 4-16 V F 4-5 V F V F ExecutiveTime(s) 10000 62.78 20000 127.92 30000 190.85 40000 261.31 50000 332.87 60000 382.86 70000 465.39 80000 526.35 90000 594.53-51 -
Executive Time(second) 700 600 500 400 300 200 100 0 10K 20K 30K 40K 50K 60K 70K 80K 90K Number of Frames 4-16 V F V C V O V F 100 10000 4-6 4-17 V C 4-6 V C V C ExecutiveTime(s) 1 13.70 2 26.75 3 39.34 4 51.50 5 62.78 6 72.57 7 81.48 8 91.75 9 100.06 10 107.42-52 -
Executive Time(second) 120 100 80 60 40 20 0 1 2 3 4 5 6 7 8 9 10 Max. Number of Changed States 4-17 V C 4-18 V O V F V C 10000 5 4-7 (V O (V O - 1)) / 2 V O - 53 -
4-7 V O V O ExecutiveTime(s) 100 105.92 200 437.67 300 970.77 400 1723.50 500 2623.35 600 3899.03 700 5299.55 800 7149.08 900 8815.81 1000 10887.53 Executive Time(second) 12000 10000 8000 6000 4000 2000 0 100 200 300 400 500 600 700 800 900 1000 Number of Objects 4-18 V O V F V O V C 100 5 4-8 4-19 - 54 -
4-8 V F V F ExecutiveTime(s) 10000 105.92 20000 218.10 30000 318.10 40000 458.67 50000 591.00 60000 657.65 70000 836.14 80000 929.26 90000 1057.10 1200 Executive Time(second) 1000 800 600 400 200 0 1K 2K 3K 4K 5K 6K 7K 8K 9K Number of Frames 4-19 V F V C V O V F 100 10000 4-9 4-20 V C - 55 -
4-9 V C V C ExecutiveTime(s) 1 6.53 2 21.54 3 44.68 4 75.01 5 105.92 6 145.25 7 183.45 8 234.65 9 278.09 10 322.93 Executive Time(second) 350 300 250 200 150 100 50 0 1 2 3 4 5 6 7 8 9 10 Max. Number of Changed States 4-20 V C VB.NET - 56 -
(VF) (VC) (VO) (VO (VF / 2) VC) (VO VF VC) 1/4 1/5 (VF) (VO) (VO) ((VO (VO - 1)) / 2) (VC) - 57 -
2D string 2D Z-string (1) (2) (3) Z-string AVIS FPI-tree AFPI-tree 3D 9D-SPA - 58 -
5-1 (D T) (S) 5-1 - 59 -
9D-SPA S D S T S TR (h D, h T, h TR ) S D > h D S T > h T S TR >h TR - 60 -
Adali, S., Candan, K. S., Chen, S., Subrahmanian, V. S., & Erol, K. (1996). The advanced video information system: Data structures and query processing. Multimedia Systems, 4(4), 172-186. Chan, Y. K., & Chang, C. C. (2001). Spatial similarity retrieval in video databases. Journal of Visual Communication and Image Representation, 12(2), 107-122. Chang, Y. I., Ann, H. Y., & Yeh, W. H. (2000). A unique-id-based matrix strategy for efficient iconic indexing of symbolic pictures. Pattern Recogn, 33(8), 1263-1276. Chang, Y. I., Yang, B. Y., & Yeh, W. H. (2001). A generalized prime-number-based matrix strategy for efficient iconic indexing of symbolic pictures. Pattern Recognition Letters, 22(6), 657-666. Chang, C. C. (1991). Spatial match retrieval of symbolic pictures. Journal of Information Science and Engineering, 7(3), 405-422. Chang, C. K., Shi, Q. Y., & Yan, C. W. (1987). Iconic indexing by 2-D strings. IEEE Transactions on Pattern Analysis Machine Intelligence, 9(3), 413-428. - 61 -
Huang, P. W., & Lee, C. H. (2004). Image database design based on 9D-SPA representation for spatial relation. IEEE Transactions on Knowledge and Data Engineering, 16(12), 1486-1496. Hsu, F. J., Lee, S. Y., & Lin, B. S. (1999). Video data indexing by 2D C-trees. Journal of Visual Languages and Computing, 9(4), 375-397. Huang, P. W., & Jean, Y. R. (1994). Using 2D C + -string as spatial knowledge representation for image database systems. Pattern Recognition, 27(9), 1249-1257. Huang, P. W., & Jean, Y. R. (1996). Spatial reasoning and similarity retrieval for image database systems based on RS-strings. Pattern Recognition, 29(12), 2103-2114. Jean, Y. R., & Chang, H. P. (2006). A dynamic visibility inference scheme based on new spatial knowledge representations from Observers perspective. Proceedings of the International Computer Symposium (ICS) (pp. 1139-1144), Taichung: Providence University. Jungert, E. (1988). Extended symbolic projections as a knowledge structure for spatial reasoning. Proceedings of the 4th International Conference on Pattern Recognition (pp. 343-351), London: Springer-Verlag. - 62 -
Koprulu, M., Cicekli, N. K., & Yazici, A. (2003). Spatio-temporal querying in video databases. Information Sciences, 160(1), 131-152. Lee, J. T., & Chiu, H. P. (2003). 2D Z-string: A new spatial knowledge representation for image databases. Pattern Recognition Letters, 24(16), 3015-3026. Lee, J. T., Chiu, H. P., & Yu, P. (2002). 3D C-string: A new spatio-temporal knowledge representation for video database systems. Pattern Recognition, 35(11), 2521-2537. Lee, J. T., Yu, P., Chiu, H. P., & Hong, R. W. (2005). 3D Z-string: A new knowledge structure to represent spatio-temporal relations between objects in a video. Pattern Recognition Letters, 26(16), 2500-2508. Lee, S. Y., & Hsu, F. J. (1990). 2D C-string: A new spatial knowledge representation for image database system. Pattern Recogn, 23(10), 1077-1087. Lee, S. Y., & Hsu, F. J. (1992). Spatial reasoning and similarity retrieval of images using 2D C-string knowledge representation. Pattern Recogn, 25(3), 305-318. - 63 -
Lee, S. Y., Hsu, F. J., & Lin, B. S. (1998). 2D C-Tree spatial representation for iconic image. Journal of Visual Languages and Computing, 10(2), 147-164. Su, J. H., Huang, Y. T., Yeh, H. H., & Tseng, V. S. (2010). Effective content-based video retrieval using pattern-indexing and matching techniques. Expert Systems with Applications, 37(7), 5068-5085. Yi, H., Rajan, D., & Chia, L. T. (2005). A motion-based scene tree for compressed video content management. Image and Vision Computing, 24(2), 131-142. - 64 -