home > kero > Documentation
Partition a list into a variety of partitions. For example, [1,2,3,4] can be partitioned into [[1,2],[3,4]].
kero.utils.utils.py
def partition_list(x, partition, do_shuffle=False,
verbose=False,
for_index=False):
return collection_of_batches , index_partition
Arguments/Return
| x |
list |
| partition |
Either integer or list of list.
Integer. If integer>0 is given, then each batch size is the size of the integer. For example partition=3 to list=[1,2,3,4,5,6,7] will yield [1,2,3],[4,5,6],[7,8]].
List of list. [ind1, ind2, …, indN] where ind is a list of indices for example, partition = [[2,1],[3]] for list=[x,y,z] will yield [[y,x],[z]]. |
| do_shuffle |
Bool. If true, shuffle the content of the partition result.
Default=False |
| verbose |
Bool False or integer
The larger the integer, the more information is printed. Set them to suitable integers for debugging.
Default=False |
| for_index |
Bool. (for internal recursion, not meant for usage)
Default=False |
| Return collection_of_batches |
List of list [batch], where each batch is a sub-list of x |
| Return
index_partition |
List of list. If not None, then this is the list of indices of x, corresponding to the output. For example if x=[a,b,c,d] and the collection_of_batches is [[b,a],[d,c]], then this index_partition is [[1,0],[3,2]] |
Example Usage 1
testUtils2.py
import kero.utils.utils as ut
import numpy as np
N=24
x1 = range(N)
x2 = ["a","b","c","d","e","f","g","h","i","j","k"]
batch_sizes = [7,4]
for x,batch_size in zip([x1,x2],batch_sizes ):
for shuffle in [False,True]:
print("size | items | indices (shuffle =", shuffle,")")
print("---------------------------------")
collection_of_batches, index_partition = ut.partition_list(x,batch_size, do_shuffle=shuffle)
total_size = 0
for y,ind in zip(collection_of_batches,index_partition):
print(" ",len(y)," : ",y , " : ", ind)
total_size = total_size + len(y)
print("---------------------------------")
print("total size = ", total_size,"\n")
print("----------using manual index---------------")
manual_index = [[1,2,3,4,5],[6,7],[0,10,9,8]]
partition=manual_index
collection_of_batches, _ = ut.partition_list(x,partition )
print("size | items | indices (shuffle =", shuffle,")")
print("---------------------------------")
for y in collection_of_batches:
print(" ",len(y)," : ",y)
The output is the following.
size | items | indices (shuffle = False )
---------------------------------
7 : [0, 1, 2, 3, 4, 5, 6] : [0, 1, 2, 3, 4, 5, 6]
7 : [7, 8, 9, 10, 11, 12, 13] : [7, 8, 9, 10, 11, 12, 13]
7 : [14, 15, 16, 17, 18, 19, 20] : [14, 15, 16, 17, 18, 19, 20]
3 : [21, 22, 23] : [21, 22, 23]
---------------------------------
total size = 24
size | items | indices (shuffle = True )
---------------------------------
7 : [13, 6, 10, 4, 22, 11, 9] : [13, 6, 10, 4, 22, 11, 9]
7 : [21, 20, 14, 8, 0, 23, 17] : [21, 20, 14, 8, 0, 23, 17]
7 : [15, 5, 3, 19, 16, 18, 12] : [15, 5, 3, 19, 16, 18, 12]
3 : [1, 7, 2] : [1, 7, 2]
---------------------------------
total size = 24
size | items | indices (shuffle = False )
---------------------------------
4 : ['a', 'b', 'c', 'd'] : [0, 1, 2, 3]
4 : ['e', 'f', 'g', 'h'] : [4, 5, 6, 7]
3 : ['i', 'j', 'k'] : [8, 9, 10]
---------------------------------
total size = 11
size | items | indices (shuffle = True )
---------------------------------
4 : ['f', 'j', 'e', 'b'] : [5, 9, 4, 1]
4 : ['c', 'g', 'd', 'a'] : [2, 6, 3, 0]
3 : ['i', 'h', 'k'] : [8, 7, 10]
---------------------------------
total size = 11
----------using manual index---------------
size | items | indices (shuffle = True )
---------------------------------
5 : ['b', 'c', 'd', 'e', 'f']
2 : ['g', 'h']
4 : ['a', 'k', 'j', 'i']
kero version: 0.6.2
. The following wrapper uses Runge-Kutta solver from scipy. The purpose of the wrapper is to compute the specified points 
.
