| blob | b1b6f09af30541b8a5af2271f7a78b3bbf1ad53f |
1 ## -*- mode: org; -*-
2 This file contains semi-structured notes about devel process
3 It is no always reflect real progres of patch queue
5 * Quota todo list
6 ** DONE Investigate XFS project id internals
7 - It is possible to replace the root of project in rename() syscal.
8 - It is not possible to disable cross project movements
9 ** Add directory tree id support to ext4 inode
10 : EXT4_MOUNT_TREE_ID
11 : EXT4_FEATURE_INCOMPAT_TREE_QUOTA
12 : #define EXT4_IOC_GET_TREEID _IOR('f', 16, unsigned int)
13 : #define EXT4_IOC_SET_TREEID _IOW('f', 17, unsigned int)
14 *** TODO Add directory tree_id support to lib2fs tools
15 1) commits:
16 - commit cf0eeec0159b1af67fad9bccce220402cf08dff0
17 - commit 88b34b1f87c2d7c3dfc9be7ff6e5d916f06d960a
18 - commit c2d4300b8a4a13d8a78b86c386f76259f23feec2
20 2) extra
21 commit 2921332fd88b843ffb828d9c18f05bfd171ace76
22 *** DONE Implement tree_id inheritance from dir to entry
23 Done in ext4_new_inode() right before vfs_quota_init
24 *** DONE Prohibit bad operations like
25 **** rename
26 *UPDATE* Due to locking issues cross-tires renames completely
27 prohibited
28 1) disallow for all types where ddir->tree\_id != sdir->tree\_id
29 2) disallow if !DIR and i\_lnk != 1
30 3) disallow if DIR and i\_lnk != 2
31 Add *FIXME_NOTE* about quota transfer in cross tree rename() [1]
33 **** link
34 allow only if src->tree\_id == dst->tree\_id
35 *** TODO Add runtime checks for inapropriate tree.
36 In fact some one may turn-off TREE_ID mount flag and then mix up tries
37 content. Later TREE_ID flag may passed on new mount (remount are not possible)
38 We have to check what tree inheritance assumption is ok.
39 **** DONE define inheritance assumptions
40 **** TODO find suitable place for checks [1/1]
41 suitable place is following where we have parent dentry and child dentry
42 The check must being done only once because it is not possible to change
43 it back.
44 Probably ext4_lookup is a good candidate for that.
45 In case of error inode will be marked as containg bad quota.
46 this may result in troubles because it may already contain quota info.
47 In this case we have to print _huge_ warning msg and drop quota.
48 - [X] ext4_lookup :: check for proper inheritance
50 *** Performance
51 **** open-close 100000, ext3 with usr+grp quota each thread run on didicated fs.
52 *Awful results*. Quota locking is fatally sequential. It is ten times slower
53 in case of 4 concurrent filesystems.
54 |-------------+-----------+------------|
55 | num threads | quota-off | w quota-on |
56 | 1 | 1792677 | 7353557 |
57 | | | |
58 | 2 | 1888589 | 8345385 |
59 | | 1954368 | 8401524 |
60 | | | |
61 | 3 | 2176776 | 11969783 |
62 | | 2300495 | 12422900 |
63 | | | |
64 | 4 | 2475488 | 20070904 |
65 | | 2651336 | 17373849 |
66 |-------------+-----------+------------|
68 **** It is strange but result depend on what user are execute tests :)
69 | test | ext3 | ext3+quota | su ext3 | su ext3+quota |
70 | open-close 1000000 | 2380660 | 2475557 | 3375653 | 3645048 |
71 | | 2380660 | 2370699 | 3359800 | 3641397 |
72 | | 2360164 | 2455658 | 3528724 | 3642408 |
73 | | | | | |
74 | open-unlink 100000 | 1749507 | 2627881 | 1786597 | 4488170 |
75 | | 1751481 | 2611456 | 1788580 | 4405862 |
76 | | 1757022 | 2629494 | 1791130 | 4438595 |
77 |--------------------+-----------+------------+----------+---------------|
79 **** Comparison chart vanila and with tree-quota patch queue.
80 EXT3
81 |--------------------+-------------+------------+---------+----------|
82 | test | su ext3 | w patch | ext3+q | w patch |
83 | open-close 1000000 | 3375653 | 2377562 | 3645048 | 2969361 |
84 | | 3359800 | 2367956 | 3641397 | 3028390 |
85 | | 3528724 | 2401222 | 3642408 | 2966994 |
86 | | | | | |
87 | open-unlink 100000 | 1786597 | 1802290 | 4488170 | 2814249 |
88 | | 1788580 | 1785768 | 4405862 | 2827910 |
89 | | 1791130 | 1777920 | 4438595 | 2813790 |
90 | | | | | |
91 | chown 100000 | 2877498 /10 | 2861473/10 | 2782447 | 3872993 |
92 | | 2855129 /10 | 2868539/10 | 2805415 | 3783275 |
93 | | 2865346 /10 | 2858689/10 | 2799819 | 3894187 |
94 | | | | | |
95 | chown2 100000 | 2854466 /10 | 2867439/10 | 6691358 | 8453543 |
96 | | 2855050 /10 | 2869318/10 | 6412669 | 7932430 |
97 | | 2894878 /10 | 2930830/10 | 6638210 | 8097675 |
98 | | | | | |
99 | chown3 100000 | 5693838 /10 | 5724797/10 | 6957357 | 12402760 |
100 | | 5688717 /10 | 5791604/10 | 6526796 | 13011283 |
101 | | 5705545 /10 | 5721885/10 | 6894547 | 12890701 |
102 |--------------------+-------------+------------+---------+----------|
103 EXT4
104 |--------------------+------------+------------+------------+----------|
105 | test | ext4 | w patch | ext4+quota | w patch |
106 | open-close 1000000 | 2572571 | 2548441 | 3831249 | 34207090 |
107 | | 2562564 | 2540445 | 3843340 | 40511598 |
108 | | 2568982 | 2570501 | 3902613 | 37835694 |
109 | | | | | |
110 | open-unlink 100000 | 2433148 | 2219039 | 4286613 | 31892179 |
111 | | 2456095 | 2228081 | 4307643 | 8150296 |
112 | | 2437422 | 2243730 | 4652567 | 19954407 |
113 | | | | | |
114 | chown 100000 | 3051037/10 | 2948017/10 | 2985798 | 6688243 |
115 | | 3041008/10 | 2935703/10 | 2984359 | 6141977 |
116 | | 3089298/10 | 2935703/10 | 3036337 | 6527266 |
117 | | | | | |
118 | chown2 100000 | 3024882/10 | 2980009/10 | 7211971 | 11540048 |
119 | | 3051953/10 | 2959488/10 | 7071395 | 10705118 |
120 | | 3018066/10 | 2966467/10 | 7000306 | 10888212 |
121 | | | | | |
122 | chown3 100000 | 6045590/10 | 5886467/10 | 6641102 | 14452187 |
123 | | 6045006/10 | 5922254/10 | 6579055 | 14987704 |
124 | | 6045006/10 | 5901298/10 | 6683252 | 15007360 |
125 |--------------------+------------+------------+------------+----------|
128 ** Add trace points to quota subsys
129 This helps in later debugging. And probably this will helps
130 later code to be easily accepted in mainstream.
131 **** Define proper trace events
132 **** Add trace_points to quota code
133 ** Add tree quota support to ext4
134 : EXT4_MOUNT_TRQUOTA
135 *** DONE This about best way to implement quota tree :NOTE:
136 In fact where are two possible ways to accomplish this task
137 **** Store inode id inside ondisk structure for each inode.
138 Implement it as xfs does.
139 **** Store tree_id only for root nodes
140 We will inherent tree\_id from parent on path_lookup and store it
141 only in memory inode structure.
142 **** Comparison
143 | Characteristics | ondisk | in memory | |
144 |---------------------+-------------------------+-----------------------------------------+---|
145 | demadge resistance | +Best(fsck) | -Only in runtime | |
146 | orphan list replay | +No problem | -Where is no parent at this time, so we | |
147 | | | have to remember tree_id some where | |
148 | Disk layout changes | -Required | +Not required | |
149 | Performance | -More disk IO | + Minimum disk activity | |
150 | | cross tree rename() [1] | | |
152 *** DONE Think about quota transfer interface
153 Currently transfer get i_gid and i_uid from "struct iattr"
154 so where is no place from i_treeid for
155 *** DONE perform quota transfer on rename
156 Currently where is no quota transfer on rename, we have to perform
157 it and if EDQUOT fail whole operation.
158 - Dont forget to mark inode as dirty because of i_tree changes.
160 *** DONE Fix journal transaction block credits
161 Most places has hardcoded usr+grp
162 *** Testing
163 **** ext4+quota [2/2]
164 - [X] fsstress in different trees
165 - [X] open-close / open-unlink / chown{,2,3}
166 - [ ] seq quotaon/quotaoff
167 **** ext4+ journaled quota [3/4]
168 - [X] fsstress in different trees
169 - [X] open-close / open-unlink / chown{,2,3}
170 - [X] seq quotaon/quotaoff
171 - [ ] concurrent quotaon, quotaoff
172 - [ ] mount / remount / concurent remount
174 **** Create testing tools [1/3]
175 - [ ] Add tree id support to fsstress.
176 - [X] add mv rename operation.
178 ** Add separate tree/usr quota id for ext4
179 This task must being done in following steps
180 *** Extend qutid_t type to 64 bit (Change all interfaces to use this bits)
181 **** TODO think about quotactl interface currently it support only 32bit id
182 *** Investigate quota file format v0
183 *** Create 64-bit quota file format
184 **** Testing
185 It must being tested with fsstress like follows on ext4:
186 : - decode id = (id << 32) + ~((int)id)
187 : - decode id = (id << 32) + tree_id
190 ** footnotes
191 Footnotes:
192 [1] Cross tree rename. Usually we have to mark srcdir and dstdir
193 as dirty, but now we have to also mark inode itself because its tree id may
194 be changed