Overview
This is a Node.js module to run the batch requests of Google APIs.
Description
In Google APIs, there are APIs where batch requests can be run. The batch requests can run multiple API calls by one API call with the asynchronous process. By this, both the process cost and the quota cost can be reduced. Ref In Node.js, the wonderful module of googleapis for Node.js is existing. But, in the current stage, unfortunately, it seems that the googleapis for Node.js cannot run the batch requests. Ref So, I created this module. This module can achieve batch requests with Node.js. In order to run batch requests, the access token retrieved from googleapis for Node.js can be used.
Gists
This is a sample script for the resumable upload using Axios with Node.js.
Sample script
In this sample script, as a sample situation in order to explain the resumable upload, the file data is loaded from the local PC, and the data is uploaded to Google Drive with the resumable upload.
const axios = require("axios");
const fs = require("fs").promises;
async function sample() {
const filepath = "./###"; // Please set the filename and file path of the upload file.
const new_access_token = "###"; // Please set your access token.
const name = "###"; // Please set the filename on Google Drive.
const mimeType = "###"; // Please set the mimeType of the uploading file. I thought that when this might not be required to be used.
// 1. Prepare chunks from loaded file data.
const split = 262144; // This is a sample chunk size.
const data = await fs.readFile(filepath);
const fileSize = data.length;
const array = [...new Int8Array(data)];
const chunks = [...Array(Math.ceil(array.length / split))].map((_) =>
Buffer.from(new Int8Array(array.splice(0, split)))
);
// 2. Retrieve endpoint for uploading a file.
const res1 = await axios({
method: "POST",
url: "https://www.googleapis.com/upload/drive/v3/files?uploadType=resumable",
headers: {
Authorization: `Bearer ${new_access_token}`,
"Content-Type": "application/json",
},
data: JSON.stringify({ name, mimeType }),
});
const { location } = res1.headers;
// 3. Upload the data using chunks.
let start = 0;
for (let i = 0; i < chunks.length; i++) {
const end = start + chunks[i].length - 1;
const res2 = await axios({
method: "PUT",
url: location,
headers: { "Content-Range": `bytes ${start}-${end}/${fileSize}` },
data: chunks[i],
}).catch(({ response }) =>
console.log({ status: response.status, message: response.data })
);
start = end + 1;
if (res2?.data) console.log(res2?.data);
}
}
sample();
Testing
When this sample script is run, the following result is obtained.
Gists
This is a sample script for uploading files to Google Drive with asynchronous process using Python.
Sample script
import aiohttp
import asyncio
import json
folder_id = "###" # Please set the folder ID you want to put.
token = "###" # Please set your access token.
url = "https://www.googleapis.com/upload/drive/v3/files"
async def workers(file):
async with aiohttp.ClientSession() as session:
metadata = {"name": file["filename"], "parents": [folder_id]}
data = aiohttp.FormData()
data.add_field("metadata", json.dumps(metadata), content_type="application/json; charset=UTF-8")
data.add_field("file", open(file["path"], "rb"))
headers = {"Authorization": "Bearer " + token}
params = {"uploadType": "multipart"}
async with session.post(url, data=data, params=params, headers=headers) as resp:
return await resp.json()
async def main():
# Please set the filenames and the file paths as follows.
fileList = [
{"filename": "sample1", "path": "./sample1.png"},
,
,
,
]
works = [asyncio.create_task(workers(e)) for e in fileList]
res = await asyncio.gather(*works)
print(res)
asyncio.run(main())
- When this script is run, the files of
fileList are uploaded to Google Drive with the asynchronous process.
Note
- This sample supposes that your access token can be used for uploading files to Google Drive using Drive API. Please be careful about this.
Reference
Gists
This is a sample script for using Google API Client Library (gapi) for JavaScript with the service account. Unfortunately, in the current stage, gapi cannot directly use the service account. So, in this case, it is required to implement the script for retrieving the access token from the service account. In this report, I would like to introduce the method for using gapi with the service account using a Javascript library.
Gists
In this report, I would like to report about the current specification of the search query for the file list method in Drive API.
Recently, I noticed that the specification of the search query for the file list method in Drive API might have been changed. I thought that to know the change of specification of the search query is important for creating the application using Drive API. In this report, I would like to introduce the current specification of the search query.
Gists
This is a simple sample script for achieving the resumable upload to Google Drive using Axios. In order to achieve the resumable upload, at first, it is required to retrieve the location, which is the endpoint of upload. The location is included in the response headers. After the location was retrieved, the file can be uploaded to the location URL.
In this sample, a text data is uploaded with the resumable upload using a single chunk.
Gists
This is a sample script of Node.js for downloading the data and uploading the data to Google Drive with the resumable upload without saving it as a file. The downloaded data is uploaded to Google Drive with the stream.
Sample script
Before you use this, please set the variables of accessToken, url, fileSize, mimeType and filename. In this case, fileSize is required to set because the data is uploaded with the resumable upload.
Updated: GetFileList for golang, Javascript, Node.js and Python
This is the libraries to retrieve the file list with the folder tree from the specific folder of own Google Drive and shared Drives.
node module - google-drive-getfilelist was updated to v1.0.4
-
v1.0.4 (May 14, 2020)
-
Shared drive got to be able to be used. The file list can be retrieved from both your Google Drive and the shared drive.
- For example, when the folder ID in the shared Drive is used
id of resource, you can retrieve the file list from the folder in the shared Drive.
You can get this from https://github.com/tanaikech/node-getfilelist
You can also get this from https://www.npmjs.com/package/google-drive-getfilelist
Gists
This is a simple sample script for achieving the resumable upload to Google Drive using Python. In order to achieve the resumable upload, at first, it is required to retrieve the location, which is the endpoint of upload. The location is included in the response headers. After the location was retrieved, the file can be uploaded to the location URL.
In this sample, a PNG file is uploaded with the resumable upload using a single chunk.
Gists
This is a simple sample script for achieving the resumable upload to Google Drive using Node.js. In order to achieve the resumable upload, at first, it is required to retrieve the location, which is the endpoint of upload. The location is included in the response headers. After the location was retrieved, the file can be uploaded to the location URL.
In this sample, a PNG file is uploaded with the resumable upload using a single chunk.
Gists
This is a sample script for creating a table to Google Document by retrieving values from Google Spreadsheet for Node.js.
Before you use this script, please install Node.js module of node-gdoctableapp.
$ npm install --save-dev gdoctableapp
or
$ npm install --global gdoctableapp
Sample script:
This sample script uses Service Account.
In this sample script, the values are retrieved from Sheet1!A1:C5 of Spreadsheet, and new table is created to the Document using the values.
Overview
This is a Node.js module to manage the tables on Google Document using Google Docs API.
Description
Google Docs API has been released. When I used this API, I found that it is very difficult for me to manage the tables on Google Document using Google Docs API. Although I checked the official document, unfortunately, I thought that it’s very difficult for me. So in order to easily manage the tables on Google Document, I created this library.
Gists
This sample script is for directly using the refreshed access token by googleapis for Node.js. When oauth2Client.refreshAccessToken((err, tokens) => {}); is used to retrieve the refreshed access token, the following error occurs.
DeprecationWarning: The refreshAccess
Token method has been deprecated, and will be removed in the 3.0 release of goo
gle-auth-library. Please use the getRequestHeaders method instead.
It is required to use getRequestHeaders(). But I couldn’t find the sample script using getRequestHeaders(). So I created this sample script. If this was useful for you, I’m glad.
node module - google-drive-getfilelist was updated to v1.0.3
You can get this from https://github.com/tanaikech/node-getfilelist
You can also get this from https://www.npmjs.com/package/google-drive-getfilelist
Overview
This is a Node.js module to retrieve the file list with the folder tree from the specific folder of Google Drive.
Description
When I create applications for using Google Drive, I often retrieve a file list from a folder in the application. So far, I had created the script for retrieving a file list from a folder for each application. Recently, I thought that if there is the script for retrieving the file list with the folder tree from the folder of Google Drive as a module, it will be useful for me and other users. So I created this.
Gists
This is a sample script for retrieving a folder tree using Node.js. In this sample, you can set the top of folder for the folder tree. In generally, the folder tree is created by retrieving folders from the top folder in order. For example, when Google Apps Script is used, the script becomes like this. But when Drive API is used for this situation, if there are a lot of folders in the top folder, a lot of APIs are required to be called. So in this sample, I have tried to create the folder tree by a small number of API calls as possible.
Gists
This is a sample script for sending e-mails from gmail using Nodemailer. In order to use this, please retrieve the folloing parameters before run this script.
- gmail address
- client ID
- client Secret
- Refresh token
- Please include
https://mail.google.com/ in the scope.
- Enable gmail API at API console.
- Install Nodemailer
const nodemailer = require('nodemailer');
var auth = {
type: 'oauth2',
user: '### your gmail address ###',
clientId: '### client ID ###',
clientSecret: '### client secret ###',
refreshToken: '### refresh token ###',
};
var mailOptions = {
from: '#####',
to: '#####',
subject: 'sample subject',
text: 'sample text',
html: '<b>sample html</b>',
};
var transporter = nodemailer.createTransport({
service: 'gmail',
auth: auth,
});
transporter.sendMail(mailOptions, (err, res) => {
if (err) {
return console.log(err);
} else {
console.log(JSON.stringify(res));
}
});
Reference :
Gists
This sample script is for updating thumbnail of file on Google Drive using Python.
This sample supposes that quickstart is used and default quickstart works fine. In order to use this sample, please carry out as follows.
- Replace
main() of the default quickstart to this sample.
Script :
import base64 # This is used for this sample.
def main():
credentials = get_credentials()
http = credentials.authorize(httplib2.Http())
service = discovery.build('drive', 'v3', http=http)
with open("./sample.png", "rb") as f:
res = service.files().update(
fileId="### file ID ###",
body={
"contentHints": {
"thumbnail": {
"image": base64.urlsafe_b64encode(f.read()).decode('utf8'),
"mimeType": "image/png",
}
}
},
).execute()
print(res)
contentHints.thumbnail.image is URL-safe Base64-encoded image. So an image data that you want to use as new thumbnail has to be converted to URL-safe Base64-encoded data. For this, it uses base64.urlsafe_b64encode() at Python.
Gists
This sample script is for downloading files under a specific folder using Node.js. It can download files with Google Docs and others.
This sample supposes as follows. So please confirm it.
- quickstart is used and default quickstart works fine.
In order to use this sample, please carry out as follows.
- Replace
listFiles() of the default quickstart to this sample.
- Set
folderid. This script can retrieve files in the folder with folderid.
- Delete
drive-nodejs-quickstart.json. I think that there is the file at .credentials in your home directory.
- Change the SCOPE from
var SCOPES = ['https://www.googleapis.com/auth/drive.metadata.readonly']; to var SCOPES = ['https://www.googleapis.com/auth/drive.readonly'];.
- Run script, retrieve the code and authorize.
Script :
function listFiles(auth) {
var folderid = "### folder ID ###"; // Folder ID. This script downloads files in the folder with this folder ID.
var outputExtension = "pdf"; // Extension of output file. This is adapted to only Google Docs.
var outputMimeType = mime.lookup(outputExtension);
var service = google.drive('v3');
service.files.list({
auth: auth,
q: "'" + folderid + "' in parents and trashed=false",
fields: "files(id, name, mimeType)"
}, function(err, response) {
if (err) {
console.log('The API returned an error: ' + err);
return;
}
response.files.forEach(function(e){
if (e.mimeType.includes("application/vnd.google-apps")) {
var dlfile = fs.createWriteStream(e.name + "." + outputExtension);
service.files.export({
auth: auth,
fileId: e.id,
mimeType: outputMimeType
}).on('end', function() {
console.log("'%s' was downloaded as %s.", e.name, outputExtension);
}).on('error', function(err) {
console.error(err);
return process.exit();
}).pipe(dlfile);
} else {
var dlfile = fs.createWriteStream(e.name);
service.files.get({
auth: auth,
fileId: e.id,
alt: 'media'
}).on('end', function() {
console.log("'%s' was downloaded as %s.", e.name, mime.extension(e.mimeType));
}).on('error', function(err) {
console.error(err);
return process.exit();
}).pipe(dlfile);
}
});
});
}
Gists
Updated: January 22, 2023
This sample script is for the interconversion between Google Docs (document, spreadsheet and presentation) and Microsoft Docs (word, excel and powerpoint). The feature is to convert them without Advanced Google Services.
Since Advanced Google Services is not used for this, if you publish your script with this script, you are not necessary to explain how to install Advanced Google Services. This script converts between Google Docs and Microsoft Docs using UrlFetchApp.fetch(). Although Drive API v3 is used, Drive API is automatically enabled by the recent update on Google. Ref (I’m sorry. This is Japanese language.) So you are not necessary to explain about the use of Drive API. By this, users will be easy to use the scripts that Drive API is used. This is very important for a lot of users.
This sample script converts from Microsoft Docx File on Google Drive to Google Spreadsheet, and converts to HTML file.
Drive APIs v2 and v3 are used for this. Please set as follows.
“Drive API v2” can be used at Google Apps Script by enabling Drive API of Advanced Google services and of Google API Console.
How to use it is as follows.
-
In the script editor, select Resources > Advanced Google services
Overview
In this article, I would like to introduce how to transfer files for Google Drive under no authorization.
This has also been published here. https://github.com/tanaikech/FileTransfer
Description
When we download and upload files for Google Drive, it usually has to use Drive API. In order to use Drive API, access token is required. If you want to make your friends download and upload files for your Google Drive, the authorization process is to take time. So I proposal this.
This is a sample script for OCR using Google Drive API. A text file which converted by OCR can be retrieved by inputting an image file.
In this sample, Python Quickstart is used. The detail information is https://developers.google.com/drive/v3/web/quickstart/python. Please read “Step 1: Turn on the Drive API” and “Step 2: Install the Google Client Library”.
from __future__ import print_function
import httplib2
import os
import io
from apiclient import discovery
from oauth2client import client
from oauth2client import tools
from oauth2client.file import Storage
from apiclient.http import MediaFileUpload, MediaIoBaseDownload
try:
import argparse
flags = argparse.ArgumentParser(parents=[tools.argparser]).parse_args()
except ImportError:
flags = None
# If modifying these scopes, delete your previously saved credentials
# at ~/.credentials/drive-python-quickstart.json
SCOPES = 'https://www.googleapis.com/auth/drive'
CLIENT_SECRET_FILE = 'client_secret.json'
APPLICATION_NAME = 'Drive API Python Quickstart'
def get_credentials():
"""Gets valid user credentials from storage.
If nothing has been stored, or if the stored credentials are invalid,
the OAuth2 flow is completed to obtain the new credentials.
Returns:
Credentials, the obtained credential.
"""
credential_path = os.path.join("./", 'drive-python-quickstart.json')
store = Storage(credential_path)
credentials = store.get()
if not credentials or credentials.invalid:
flow = client.flow_from_clientsecrets(CLIENT_SECRET_FILE, SCOPES)
flow.user_agent = APPLICATION_NAME
if flags:
credentials = tools.run_flow(flow, store, flags)
else: # Needed only for compatibility with Python 2.6
credentials = tools.run(flow, store)
print('Storing credentials to ' + credential_path)
return credentials
def main():
credentials = get_credentials()
http = credentials.authorize(httplib2.Http())
service = discovery.build('drive', 'v3', http=http)
imgfile = 'sample.png' # Image with texts (png, jpg, bmp, gif, pdf)
txtfile = 'output.txt' # Text file outputted by OCR
mime = 'application/vnd.google-apps.document'
res = service.files().create(
body={
'name': imgfile,
'mimeType': mime
},
media_body=MediaFileUpload(imgfile, mimetype=mime, resumable=True)
).execute()
downloader = MediaIoBaseDownload(
io.FileIO(txtfile, 'wb'),
service.files().export_media(fileId=res['id'], mimeType="text/plain")
)
done = False
while done is False:
status, done = downloader.next_chunk()
service.files().delete(fileId=res['id']).execute()
print("Done.")
if __name__ == '__main__':
main()
This is a sample script for converting a PDF file to a TXT file. 2 steps are required for this.
- Upload a PDF file as a Google Document
- Download a Google Document as a TXT file
In this sample, Python Quickstart is used. The detail information is https://developers.google.com/drive/v3/web/quickstart/python. Please read “Step 1: Turn on the Drive API” and “Step 2: Install the Google Client Library”.
from __future__ import print_function
import httplib2
import os
import io
from apiclient import discovery
from oauth2client import client
from oauth2client import tools
from oauth2client.file import Storage
from apiclient.http import MediaFileUpload, MediaIoBaseDownload
try:
import argparse
flags = argparse.ArgumentParser(parents=[tools.argparser]).parse_args()
except ImportError:
flags = None
SCOPES = 'https://www.googleapis.com/auth/drive'
CLIENT_SECRET_FILE = 'client_secret.json'
APPLICATION_NAME = 'Drive API Python Quickstart'
def get_credentials():
credential_path = os.path.join("./", 'drive-python-quickstart.json')
store = Storage(credential_path)
credentials = store.get()
if not credentials or credentials.invalid:
flow = client.flow_from_clientsecrets(CLIENT_SECRET_FILE, SCOPES)
flow.user_agent = APPLICATION_NAME
if flags:
credentials = tools.run_flow(flow, store, flags)
else: # Needed only for compatibility with Python 2.6
credentials = tools.run(flow, store)
print('Storing credentials to ' + credential_path)
return credentials
def main():
credentials = get_credentials()
http = credentials.authorize(httplib2.Http())
service = discovery.build('drive', 'v3', http=http)
pdffile = 'sample.pdf'
txtfile = 'sample.txt'
mime = 'application/vnd.google-apps.document'
res = service.files().create(
body={
'name': pdffile,
'mimeType': mime
},
media_body=MediaFileUpload(pdffile, mimetype=mime, resumable=True)
).execute()
dl = MediaIoBaseDownload(
io.FileIO(txtfile, 'wb'),
service.files().export_media(fileId=res['id'], mimeType="text/plain")
)
done = False
while done is False:
status, done = dl.next_chunk()
print("Done.")
if __name__ == '__main__':
main()
There are a lot of APIs on Google. When we use Google Drive APIs, they usually have “fields” as a resource. The parameter “fields” gives various information which is selected to us. This is one of important parameters. And this can be used at Google Apps Script (GAS) although that version is v2. About how to use it, there are some documents. But it is difficult to find how to use it at GAS. So I would like to write down here as a memorandum. Most parameters for APIs have to be expressed as JSON. However, the expressions are different for each API. I would like to introduce this using some samples. This is for GAS.
Overview
In this article, files can be downloaded without authorization.
Description
When we download files from Google Drive, it usually has to use Drive API. In order to use Drive API, access token is required. If you want to make your friends download files from your Google Drive, the authorization process is to take time. Also Web Link for each files can be used. But it has to set for each files. So I proposal this.
These GASs retrieve an access token for using Google Drive API. There are 3 parts. Before you use this, please retrieve client ID, client secret and redirect uri from Google , and choose scopes.
1. Retrieving code from web
This is a script to output URL for retrieving “code” from web. Please retrieve “code” by import this URL to your browser. After you run this script, using “url” got from this script, it retrieves “code”.
It is necessary to retrieve access token on Google. Scope is as follows.
https://www.googleapis.com/auth/drive
Other mimetypes can be seen here.
Download and convert from Spreadsheet to Excel
curl -X GET -sSL \
-H "Authorization: Bearer [Your access token]" \
-o "Excel file name" \
"https://www.googleapis.com/drive/v3/files/[File ID]/export?mimeType=application/vnd.openxmlformats-officedocument.spreadsheetml.sheet"
Upload and convert from Excel to Spreadsheet
curl -X POST -sSL \
-H "Authorization: Bearer [Your access token]" \
-F "metadata={ \
name : '[File name on Google Drive]', \
mimeType : 'application/vnd.google-apps.spreadsheet' \
};type=application/json;charset=UTF-8" \
-F "file=@[Your Excel file];type=application/vnd.ms-excel" \
"https://www.googleapis.com/upload/drive/v3/files?uploadType=multipart"