I am loading a bunch of arrays and putting them into a single array. The loading of the data is negligible, but putting it into
combinedArray takes significant time (as in this code).
import time import numpy as np start_time = time.time() st,en = 0,0 combinedArray = np.empty(shape = (30000*30000)) #Get the array that each process wrote to for i in range(10): #load the array #ex loading X_np = np.empty(shape = (30000*30000)/10) st = en en += np.shape(X_np[:]) #add to the array list combinedArray[st:en] = X_np[:] print("Array Calc:",time.time()-start_time)
What I have found is often someone talking about not using append, so I tried with creating the array first, but just moving it is time consuming. Any advice on how to optimize this is appreciated.