如何使用 angular 和 nodejs 将 csv 数据转储到 mongoDB
Posted
技术标签:
【中文标题】如何使用 angular 和 nodejs 将 csv 数据转储到 mongoDB【英文标题】：how to dump csv data into mongoDB using angular and nodejs 【发布时间】：2018-08-16 04:46:16 【问题描述】：我有一个 csv 文件，想使用 angular 和 nodejs 将其中的数据转储到 mongodb。
需要帮助使用 angular 读取 csv 文件的数据并解析它并存储在 mongodb 中
import Injectable from '@angular/core';
import Http from '@angular/http';
import 'rxjs/add/operator/map';
@Injectable()
export class TemplateService
domain = 'http://localhost:3000';
constructor(private http: Http)
postCsvToMongo(csvJson: any)
console.log('CSV!!!' + csvJson);
return this.http.post(this.domain + '/fu/fileupload', csvJson).map(res => res.json());
这是我用来通过 post 路由发送 json 对象的代码。
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Error</title>
</head>
<body>
<pre>TypeError: Converting circular structure to JSON
<br> at JSON.stringify (<anonymous>)
<br> at router.post (/home/agam/projects/AMEX/routes/fileUpload.js:10:23)
<br> at Layer.handle [as handle_request] (/home/agam/projects/AMEX/node_modules/express/lib/router/layer.js:95:5)
<br> at next (/home/agam/projects/AMEX/node_modules/express/lib/router/route.js:137:13)
<br> at Route.dispatch (/home/agam/projects/AMEX/node_modules/express/lib/router/route.js:112:3)
<br> at Layer.handle [as handle_request] (/home/agam/projects/AMEX/node_modules/express/lib/router/layer.js:95:5)
<br> at /home/agam/projects/AMEX/node_modules/express/lib/router/index.js:281:22
<br> at Function.process_params (/home/agam/projects/AMEX/node_modules/express/lib/router/index.js:335:12)
<br> at next (/home/agam/projects/AMEX/node_modules/express/lib/router/index.js:275:10)
<br> at Function.handle (/home/agam/projects/AMEX/node_modules/express/lib/router/index.js:174:3)
<br> at router (/home/agam/projects/AMEX/node_modules/express/lib/router/index.js:47:12)
<br> at Layer.handle [as handle_request] (/home/agam/projects/AMEX/node_modules/express/lib/router/layer.js:95:5)
<br> at trim_prefix (/home/agam/projects/AMEX/node_modules/express/lib/router/index.js:317:13)
<br> at /home/agam/projects/AMEX/node_modules/express/lib/router/index.js:284:7
<br> at Function.process_params (/home/agam/projects/AMEX/node_modules/express/lib/router/index.js:335:12)
<br> at next (/home/agam/projects/AMEX/node_modules/express/lib/router/index.js:275:10)
</pre>
</body>
</html>这是我在尝试调试通过节点端的发布请求收到的请求时遇到的错误。 以及如何通过后端接收文件？？
var csv = require('fast-csv');
var mongoose = require('mongoose');
var Bulletin = require('../models/bulletin');
var multer = require("multer");
module.exports = (router) =>
router.post('/fileupload', (req, res) =>
console.log('@@@ ' + req);
res.send(req.files);
);
/*router.post('/fileupload', multer(dest: "./uploads/").array("uploads", 12), function(req, res)
res.send(req.files);
);*/
return router;
;这是nodejs代码
1","Fed official says weak data caused by weather, should not slow taper","http://www.latimes.com/business/money/la-fi-mo-federal-reserve-plosser-stimulus-economy-20140310,0,1312750.story\?track=rss","Los Angeles Times","b","www.latimes.com","1394470370698
2","Fed's Charles Plosser sees high bar for change in pace of tapering","http://www.livemint.com/Politics/H2EvwJSK2VE6OF7iK1g3PP/Feds-Charles-Plosser-sees-high-bar-for-change-in-pace-of-ta.html","Livemint","b","www.livemint.com","1394470371207
3","US open: Stocks fall after Fed official hints at accelerated tapering","http://www.ifamagazine.com/news/us-open-stocks-fall-after-fed-official-hints-at-accelerated-tapering-294436","IFA Magazine","b","www.ifamagazine.com","1394470371550
4","Fed risks falling 'behind the curve', Charles Plosser says","http://www.ifamagazine.com/news/fed-risks-falling-behind-the-curve-charles-plosser-says-294430","IFA Magazine","b","www.ifamagazine.com","1394470371793
5","Fed's Plosser: Nasty Weather Has Curbed Job Growth","http://www.moneynews.com/Economy/federal-reserve-charles-plosser-weather-job-growth/2014/03/10/id/557011","Moneynews","b","www.moneynews.com","1394470372027
6","Plosser: Fed May Have to Accelerate Tapering Pace","http://www.nasdaq.com/article/plosser-fed-may-have-to-accelerate-tapering-pace-20140310-00371","NASDAQ","b","www.nasdaq.com","1394470372212这是我的示例 csv 文件
> amex@1.0.0 start /home/agam/projects/AMEX
> node index.js
(node:6478) DeprecationWarning: current URL string parser is deprecated, and will be removed in a future version. To use the new parser, pass option useNewUrlParser: true to MongoClient.connect.
listening on port 3000
Conected to database: amex
@@@ [object Object]这是它正在打印的东西
// 1) Express for nodejs
const express = require('express');
const app = express();
const template = require('./routes/template');
const upload = require('./routes/upload');
const path = require('path');
// const fileUpload = require('express-fileupload');
const router = express.Router();
const fu = require('./routes/fileUpload')(router);
//const carts = require('./routes/carts')(router);
// 5) body parser
const bodyParser = require('body-parser');
const cors = require('cors'); // CORS is a node.js package for providing a Connect/Express middleware that can be used to enable CORS with various options.
// 2) Mongodb connection
var mongoose = require('mongoose');
const config = require('./config/database');
mongoose.connect(config.uri, (err) =>
if (err)
console.log('error ' + err);
else
console.log('Conected to database: ' + config.db);
);
app.use(cors( origin: 'http://localhost:4200' ));
// 5) put bodyparser code before the routes (converts request to req.body)
app.use(bodyParser.urlencoded( extended: true ));
app.use(bodyParser.json());
app.use(function(req, res, next)
res.header("Access-Control-Allow-Origin", "*");
res.header('Access-Control-Allow-Methods', 'PUT, GET, POST, DELETE, OPTIONS');
res.header("Access-Control-Allow-Headers", "Origin, X-Requested-With, Content-Type, Accept");
next();
);
// 3) connect node.js to angular
app.use(express.static(__dirname + '/client/dist/'));
// app.use(fileUpload());
app.use('/fu', fu); // Use Product routes in application
//app.use('/carts', carts);
//
app.get('*', (req, res) =>
res.sendFile(path.join(__dirname + '/client/dist/index.html'));
);
app.listen(3000, () =>
console.log('listening on port 3000');
);这是你期待的我的 index.js 文件
【问题讨论】：
您是否能够以角度读取 csv 文件并将其转换为 json ，因为这似乎是显而易见的方法。 其实我不知道方法，甚至不知道如何读取角度的csv文件并将其转换为json。请帮助我，因为我是这个领域的新手并正在学习 stackblitz.com/edit/angular-mvq5ju 请使用此链接访问代码。我已经在 stackblitz.com 上创建了项目请使用上面的链接 请任何帮助！！！！ @NicholasKyriakides 请在stackblitz.com/edit/angular-mvq5ju上找到该项目 【参考方案1】：您好，我在您的 StackBiltz 示例中做了一些更改，因此我能够通过 post 调用以 JSON 数组的形式发布有效负载，并创建一个接受此有效负载的服务订阅将有效负载传递到您的节点发布 api
创建了一个名为 pass 的函数
test.component.ts
import Component, OnInit from '@angular/core';
import Router from "@angular/router";
import FileUtil from './file.util';
import Constants from './test.constants';
import TestService from './test.service';
@Component(
selector: 'app-test',
templateUrl: './test.component.html',
styleUrls: ['./test.component.css']
)
export class TestComponent implements OnInit
fileImportInput: any;
data;
csvRecords:any = [];
constructor(private _router: Router,
private _fileUtil: FileUtil,private testService:TestService
)
ngOnInit()
// METHOD CALLED WHEN CSV FILE IS IMPORTED
fileChangeListener($event): void
var text = [];
var target = $event.target || $event.srcElement;
var files = target.files;
if (Constants.validateHeaderAndRecordLengthFlag)
if (!this._fileUtil.isCSVFile(files[0]))
alert("Please import valid .csv file.");
this.fileReset();
var input = $event.target;
var reader = new FileReader();
reader.readAsText(input.files[0]);
reader.onload = (data) =>
let csvData = reader.result;
let csvRecordsArray = csvData.split(/\r\n|\n/);
var headerLength = -1;
if (Constants.isHeaderPresentFlag)
let headersRow = this._fileUtil.getHeaderArray(csvRecordsArray, Constants.tokenDelimeter);
headerLength = headersRow.length;
this.csvRecords = this._fileUtil.getDataRecordsArrayFromCSVFile(csvRecordsArray,
headerLength, Constants.validateHeaderAndRecordLengthFlag, Constants.tokenDelimeter);
this.passThisFunction(this.csvRecords);
console.log('this.csvRecords',this.csvRecords);
if (this.csvRecords == null)
//If control reached here it means csv file contains error, reset file.
this.fileReset();
reader.onerror = function ()
alert('Unable to read ' + input.files[0]);
;
;
fileReset()
this.fileImportInput.nativeElement.value = "";
this.csvRecords = [];
passThisFunction(records:any)
console.log('inside function call');
console.log('records',records);
let json=JSON.stringify(records);
console.log('jsonified',json);
var str = json.replace(/,/g, ",\r\n");
console.log('das' + str);
this.testService.postCsvToMongo(str).subscribe((data) =>
console.log(data);
);
test.component.html
<h4>Read .CSV File using Angular2, JQuery and Bootstrap</h4>
<table>
<tr>
<td>
<input type="file"
#fileImportInput
name="File Upload"
id="txtFileUpload"
class="btn btn-primary"
(change)="fileChangeListener($event)"
accept=".csv"
/>
</td>
</tr>
</table>
<div>
<table class="table table-responsive table-hover" border="1"
style="width : 50%;">
<tbody>
<ng-container >
<tr >
<td *ngFor="let csvRec of csvRecords;let i=index">csvRec</td>
</tr>
</ng-container>
</tbody>
</table>
</div>
file.util.ts
import Injectable from '@angular/core';
@Injectable()
export class FileUtil
constructor()
isCSVFile(file)
return file.name.endsWith(".csv");
getHeaderArray(csvRecordsArr, tokenDelimeter)
let headers = csvRecordsArr[0].split(tokenDelimeter);
let headerArray = [];
for (let j = 0; j < headers.length; j++)
headerArray.push(headers[j]);
return headerArray;
validateHeaders(origHeaders, fileHeaaders)
if (origHeaders.length != fileHeaaders.length)
return false;
var fileHeaderMatchFlag = true;
for (let j = 0; j < origHeaders.length; j++)
if (origHeaders[j] != fileHeaaders[j])
fileHeaderMatchFlag = false;
break;
return fileHeaderMatchFlag;
getDataRecordsArrayFromCSVFile(csvRecordsArray, headerLength,
validateHeaderAndRecordLengthFlag, tokenDelimeter)
console.log(csvRecordsArray);
console.log(headerLength);
console.log(validateHeaderAndRecordLengthFlag);
console.log(tokenDelimeter);
var dataArr:any = [];
for (let i = 0; i < csvRecordsArray.length; i++)
let data = csvRecordsArray[i].split(tokenDelimeter);
if(validateHeaderAndRecordLengthFlag && data.length != headerLength)
if(data=="")
alert("Extra blank line is present at line number "+i+", please remove it.");
return null;
else
alert("Record at line number "+i+" contain "+data.length+" tokens, and is not matching with header length of :"+headerLength);
return null;
dataArr=data;
console.log('dataArr',dataArr);
return dataArr;
test.service.ts
importInjectable from '@angular/core';
importHttp from '@angular/http';
import map from 'rxjs/operators';
@Injectable()
export class TestService
API_URL:string='your api url';
constructor(private http:Http)
postCsvToMongo(csvJson:any)
return this.http.post(this.API_URL,csvJson)
.pipe(map((res)=>console.log(res)));
【讨论】：
感谢您的帮助。我能够形成 json 对象，但是当我发送 post 路由时，它给出了错误的请求错误 通过 post 路由发送 json 对象时出错 你能分享你的节点api调用是什么错误？和节点服务器文件代码？以上是关于如何使用 angular 和 nodejs 将 csv 数据转储到 mongoDB的主要内容，如果未能解决你的问题，请参考以下文章
如何使用使用 Angular 2 发布的 nodejs 保存包含图像的表单数据
如何使用 Mongoose、Express、Angular 4 和 NodeJS 上传/保存和显示图片
如何使用 keyCloak 保护 Angular（accessType-Public）和 Nodejs 应用程序（accesType-bearer-only）
angular2-webpack-starter 如何将 NodeJS 服务器添加到项目中
集成 Angular + Nodejs + Spring Boot Java REST API 使 Angular Universal 工作