Orientação gerada a partir de informações públicas. Apenas para referência.
Fast CSV é uma biblioteca de análise de CSV de alto desempenho para Dart, projetada para leitura e escrita eficientes de arquivos CSV em múltiplas plataformas. Suporta análise padrão de CSV com boa velocidade e inclui streaming baseado em eventos experimentais para lidar com arquivos grandes sem uso excessivo de memória. Ideal para aplicações que precisam de processamento confiável e rápido de CSV, como importação/exportação de dados, análises ou carregamento de configurações. O pacote é leve e se integra perfeitamente em projetos Flutter.
Casos de uso
Importação de dados de arquivos CSV
Exportando dados para formato CSV
Processamento de grandes conjuntos de dados CSV
Lendo arquivos de configuração
Analisando dados semelhantes a planilhas
Integração com fontes de dados do backend
Principais recursos
Análise de alta velocidade
Parser experimental de streaming
Suporte multiplataforma
API simples para uso básico
Leve e eficiente
Suporta codificação UTF-8
Indicado para
Aplicativos Flutter que precisam de manipulação de CSV
Serviços de backend processando dados CSV
Aplicações para desktop e web
Processamento de arquivos grandes com baixa memória
Aplicações que exigem ingestão rápida de dados
Desenvolvedores que priorizam o desempenho
Considerações
Recursos de streaming são experimentais
Opções limitadas de formatação avançada
Nenhuma geração de CSV embutida (apenas análise)
Não otimizado para estruturas CSV muito complexas
Requer manipulação manual de entrada malformada
Nenhuma integração com interface gráfica de forma nativa
README original
Snapshot do README original do repositório-fonte vinculado. Consulte o repositório para o conteúdo mais recente.
fast_csv
Classic CSV parsers suitable for most use cases. Pretty fast parsing. With experimental event-based streaming parsing.
Version: 0.2.11
About testing generated parsers
Both parsers have been tested in both operating modes, synchronous and asynchronous modes.
For testing, files from the following source are used: CSV is a data directory which contains examples of CSV files, a flat file format describing values in a table.
https://people.sc.fsu.edu/~jburkardt/data/csv/csv.html
All files are parsed except for the malformed file mlb_players.csv.
It has unclosed quote on row 1036.
Example of the parser usage
An example of a simple way to parse 1000K rows (53M UTF-16 code units) in chunks (by ~0.53M UTF-16 code units) using events without consuming a lot of memory (during parsing) and simultaneously saving this parsed data into a virtual database (by 10K rows per transaction).
import'dart:async';
import'package:fast_csv/csv_converter.dart';
Future<void> main(List<String> args) async {
_exampleParseString();
await_exampleParseStreamWithEvents();
}
const _data = '''
Year,Make,Model,Description,Price
1997,Ford,E350,"ac, abs, moon",3000.00
1999,Chevy,"Venture ""Extended Edition""","",4900.00
1999,Chevy,"Venture ""Extended Edition, Very Large""","",5000.00
1996,Jeep,Grand Cherokee,"MUST SELL!
air, moon roof, loaded",4799.00
''';
Stream<String> _createStream() {
// Create the stream with 1000000 rowsconst count = 1000 * 1000;
final controller = StreamController<String>();
final sink = controller.sink;
const row = '1999,Chevy,"Venture В«Extended EditionВ»","",4900.00';
const rowsInChunk = count ~/ 100;
final chunk = List.generate(rowsInChunk, (i) => row).join('\n');
print('Total data amount ${row.length * count} UTF-16 code units.');
print('The data will arrive in ${chunk.length} UTF-16 code unit chunks.');
var i = 0;
Timer.periodic(Duration.zero, (timer) {
sink.add(chunk);
i += rowsInChunk;
if (i < count) {
sink.add('\n');
}
if (i >= count) {
controller.close();
timer.cancel();
}
});
return controller.stream;
}
Future<void> _exampleParseStreamWithEvents() async {
print('=========================');
print('Start streaming parsing with events');
// Get external datafinal stream = _createStream();
final sw = Stopwatch();
sw.start();
final parser = _MyParser(() async {
print('Saving to virtual database complete in ${sw.elapsed}');
sw.stop();
});
await stream.transform(CsvConverter(parser: parser)).first;
}
void_exampleParseString() {
print('=========================');
print('Parsing string');
final result = CsvConverter().convert(_data);
print(result.join('\n'));
for (var i = 1; i < result.length; i++) {
final row = result[i];
final car = row[1];
final price = num.parse(row[4]);
print('$car$price');
}
}
class_MyParserextendsCsvParser{
finalFuture<void> Function()? onComplete;
int _count = 0;
int _totalCount = 0;
int _transactionCount = 0;
finalList<List<String>> _rows = [];
_MyParser([this.onComplete]);
@overridevoidbeginEvent(CsvParserEvent event) {
if (event == CsvParserEvent.startEvent) {
_count = 0;
_totalCount = 0;
_transactionCount = 0;
_rows.clear();
}
}
@override
R? endEvent<R>(CsvParserEvent event, R? result, bool ok) {
voidsaveRows(bool isLast) {
final rows = _rows.toList();
_rows.clear();
Timer.run(() async {
// Asynchronous saving to the database.await_saveToDatabase(rows, isLast);
});
}
if (ok) {
switch (event) {
caseCsvParserEvent.rowEvent:
final row = result asList<String>;
_rows.add(row);
if (_rows.length > 10000) {
saveRows(false);
}
// Free memory
result = const <String>[] as R;
break;
caseCsvParserEvent.startEvent:
saveRows(true);
if (onComplete != null) {
Timer.run(onComplete!);
}
// Completely freeing memory from the entire list
result = const <List<String>>[] as R;
default:
}
}
return result;
}
Future<void> _saveToDatabase(List<List<String>> rows, bool isLast) async {
_transactionCount++;
_count += rows.length;
_totalCount += rows.length;
if (_count > 100000 || isLast) {
print(
'Saved to virtual database $_totalCount row(s) in $_transactionCount transaction(s)');
_count = 0;
}
if (isLast) {
print(
'Totally saved to virtual database $_totalCount row(s) in $_transactionCount transaction(s)');
}
}
}
Output:
=========================
Parsing string
[Year, Make, Model, Description, Price]
[1997, Ford, E350, ac, abs, moon, 3000.00]
[1999, Chevy, Venture"Extended Edition", , 4900.00]
[1999, Chevy, Venture"Extended Edition, Very Large", , 5000.00]
[1996, Jeep, GrandCherokee, MUST SELL!
air, moon roof, loaded, 4799.00]
Ford3000.0Chevy4900.0Chevy5000.0Jeep4799.0
=========================
Start streaming parsing with events
Total data amount 52000000 UTF-16 code units.
The data will arrive in529999 UTF-16 code unit chunks.
Saved to virtual database 100010row(s) in10transaction(s)
Saved to virtual database 200020row(s) in20transaction(s)
Saved to virtual database 300030row(s) in30transaction(s)
Saved to virtual database 400040row(s) in40transaction(s)
Saved to virtual database 500050row(s) in50transaction(s)
Saved to virtual database 600060row(s) in60transaction(s)
Saved to virtual database 700070row(s) in70transaction(s)
Saved to virtual database 800080row(s) in80transaction(s)
Saved to virtual database 900090row(s) in90transaction(s)
Saved to virtual database 1000000row(s) in100transaction(s)
Totally saved to virtual database 1000000row(s) in100transaction(s)
Saving to virtual database complete in0:00:04.087358
An example of using a configurable parser
This parser is slightly slower than the non-configurable parser.
The difference between using a normal parser and using a configurable parser is that you can specify a field separator.
Any value (such as a space or semicolon).
import'package:fast_csv/csv_ex_converter.dart';
voidmain(List<String> args) {
final parser = CsvExParser(separator: ';');
final result = CsvExConverter(parser: parser).convert(_data);
print(result.join('\n'));
for (var i = 1; i < result.length; i++) {
final row = result[i];
final car = row[1];
final price = num.parse(row[4]);
print('$car$price');
}
}
const _data = '''
Year;Make;Model;Description;Price
1997;Ford;E350;"ac, abs, moon";3000.00
1999;Chevy;"Venture ""Extended Edition""";"";4900.00
1999;Chevy;"Venture ""Extended Edition, Very Large""";"";5000.00
1996;Jeep;Grand Cherokee;"MUST SELL!
air, moon roof, loaded";4799.00
''';
About the implementation of parsers
Parsers are generated from PEG grammars.
Software used to generate parsers
Below is the source code for one of the grammars.