Hello, Everyone
■Background
I am developing an app with Atlassian Connect Express. It downloads attachment from confluence cloud’page. And I deployed my source code in Google Cloud RUN.
I used AXIOS with node.js for file downloading from Confluence Cloud, when the file’s size is greater than 100M,download always failed with “aborted” error.
Attachment: audio file (mp3,wav,m4a…)
■What I did
Read response data in chunks. I found that a request aborted exception occurred when it reached the end. The response data was not read completely. By the way my code can download successfully more than 100M from other service.
■Code
↓source code↓
const writer = fs.createWriteStream(filepath);
const response = await Axios({
url: downloadUrl,
method: "GET",
responseType: "stream",
validateStatus : null,
httpAgent: new http.Agent({ keepAlive: true }),
httpsAgent: new https.Agent({ keepAlive: true })
}).catch(err => {
console.log("axios error: "+ err);
});
const contentLength = response.headers['content-length'];
let chunklength = 0;
response.data.on("data", (chunk) => {
chunklength = chunklength + chunk.length;
console.log("chunk-length-sum: "+ chunklength);
writer.write(chunk);
});
response.data.on("error", async (err) => {
console.log("axios on error: " + err);
});
response.data.on("aborted", () => {
console.log("axios error aborted ");
});
response.data.on("close", () => {
console.log("axios close ");
});
response.data.on("end", () => {
console.log("axios end ");
});
writer.on("finish", async () => {
console.log("writer finish");
})
writer.on("error", async (err) => {
console.log("axios writer onerror: " + err);
});
response.data.on("finish", async () => {
console.log("response data onfinish");
})
writer.on("finish", async () => {
console.log("writer onfinish");
})
■Code Error Explanation
After the log reports an error, it will got to the ‘aborted’ event, and then directly go to the ‘close’ event, skipping the ‘finish’ event, resulting in interruption of downloading files.
↓log↓
......
chunk-length-sum: 108453598
......
chunk-length-sum: 109240030
chunk-length-sum: 109256414
axios error aborted
axios close
axios onerror: Error: aborted
■What I expect
Used the AXIOS with node.js can download large files or other methods can be used to download large files.
■Environment
node.js 18.7
AXIOS 0.26.2
I also try other version’s AXIOS.
In older version, error log is like this.
Error: aborted at connResetException (node:internal/errors:704:14) at TLSSocket.socketCloseListener (node:_http_client:441:19) at TLSSocket.emit (node:events:525:35) at TLSSocket.emit (node:domain:489:12) at node:net:757:14 at TCP.done (node:_tls_wrap:583:7)
Emitted 'error' event on Request instance at:
at Request.onerror (node:internal/streams/legacy:62:12)
at Request.emit (node:events:513:28)
at Request.emit (node:domain:489:12)
at IncomingMessage.<anonymous> (/usr/src/app/node_modules/request/request.js:1079:12)
at IncomingMessage.emit (node:events:513:28)
at IncomingMessage.emit (node:domain:489:12)
at emitErrorNT (node:internal/streams/destroy:151:8)
at emitErrorCloseNT (node:internal/streams/destroy:116:3)
at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {
code: 'ECONNRESET'
}
■Question
1)Does Confluence Cloud have some limitations?
2)What could be the possible reason for this error （aborted）.
3)I can’t see application logs from confluence cloud, please tell me to solve this issue what shall I do.
Any feedback, suggestions or comments would be much appreciated.
Thank you very much