You can try the following approach mentioned in the help article to improve performance. Client/server grid performance optimizations
Also you can try using CutomPaging which fetch only a fixed number of records and perform operations on this limited set of data.