G Suite Reports collector

[ 1 Service description ] [ 2 Data source description ] [ 3 Setup ] [ 4 Run the collector ] [ 5 G Suite Report lag times ] [ 6 Disclaimer ]

Service description

The G Suite Reports API is used to gain insights on content management with Google activity, audit administrator actions, and generate customer and user usage reports.

Data source description

The G Suite API generates account activities for these applications and sources. The collector process the Google API responses and send them to Devo platform that will categorize all information received on tables along rows and columns on your Devo domain.

G Suite Reports

Listed in the table below are some application names, details and how Devo platform treats the data.

Application name	Details	Devo data tables

Application name	Details	Devo data tables
Access Transparency	Activity events from a G Suite resource accessed by Google.	`cloud.gsuite.reports.access_transparency`
Admin	Report returns information on the Admin console activities of all of your account's administrators.	`cloud.gsuite.reports.admin`
Calendar	Report returns information about how your account's users manage and modify their Google Calendar events.	`cloud.gsuite.reports.calendar`
Google Drive	Report returns information about how your account's users manage, modify, and share their Google Drive documents.	`cloud.gsuite.reports.drive`
Google Cloud Platform	Activity events Interaction with the Cloud OS Login API.	`cloud.gsuite.reports.gcp`
Groups	Activity report returns information about how your account's users manage and modify their groups.	`cloud.gsuite.reports.groups`
Google+	Activity report returns information about the Google+ activity of all of your account's users.	`cloud.gsuite.reports.gplus`
Enterprise Groups	Audit activity events from actions performed by the moderator.	`cloud.gsuite.reports.cloud.gsuite.reports.groups_enterprise`
Jamboard	Activity of interactive whiteboard.	`cloud.gsuite.reports.jamboard`
Meet	Hangouts Meet Audit activity events describing a single Hangouts endpoint.	`cloud.gsuite.reports.meet`
Logins	Activity report returns information about the login activity of all of your account's users.	`cloud.gsuite.reports.login`
Mobile Audit	Activity report returns information on all activities in a mobile device with a Work account, managed by Google Mobile Management.	`cloud.gsuite.reports.mobile`
SAML	Audit activity events from login event type.	`cloud.gsuite.reports.saml`
Authorization Tokens	Activity report returns information about third-party websites and applications your users have granted access to.	`cloud.gsuite.reports.token`
Rules	Activity report returns information about how the rules (that have been set up in Admin console) are performing.	`cloud.gsuite.reports.rules`
Users Account	User Accounts Audit activity events.	`cloud.gsuite.reports.users_account`

Each report uses the basic endpoint request with report-specific parameters or event type. The maximum time period for each report is the last 180 days. For more references about G Suite Reports, visit the Google API Reference documentation.

Setup

The GSuite Reports collector requires that the Google Admin ADK API is enabled, a set of Google Oauth credentials and a token.json file for authorizing requests to the GSuite Reports API.

The following steps must be completed:

Enable the Google Admin SDK

Login to Google API console.
Select Enable APIs and Services.
In the search box enter “Admin SDK API“.
Select the Admin SDK API card in the search results.
Click Enable.

Create credentials

Follow these steps to create credentials:

While logged in Google API console. Verify that you are in the correct Google project for your GSuite account.
Go to Security > +Create Credentials > Oath Client ID.
Select Desktop app as the application type. Set any value in the Name field and click on the Create button.
A popup opens, click OK.

Download the credentials.json file for the API Client

Download the credentials in JSON format by clicking on the downward black arrow in the right corner.
Renate the file to credentials.json. You will need it for the next steps.

Authorize Oauth scopes and generate a token.json file

This step is completed using a Google-provided script. You can run this script from any computer.

from __future__ import print_function

import base64
import os.path
from googleapiclient.discovery import build
from google_auth_oauthlib.flow import InstalledAppFlow
from google.auth.transport.requests import Request
from google.oauth2.credentials import Credentials

# If modifying these scopes, delete the file token.json.
SCOPES = ['https://www.googleapis.com/auth/admin.reports.audit.readonly',
          'https://www.googleapis.com/auth/admin.reports.usage.readonly']

def main():
    """Shows basic usage of the Admin SDK Reports API.
    Prints the time, email, and name of the last 10 login events in the domain.
    """
    creds = None
    # The file token.json stores the user's access and refresh tokens, and is
    # created automatically when the authorization flow completes for the first
    # time.
    if os.path.exists('token.json'):
        creds = Credentials.from_authorized_user_file('token.json', SCOPES)
    # If there are no (valid) credentials available, let the user log in.
    if not creds or not creds.valid:
        if creds and creds.expired and creds.refresh_token:
            creds.refresh(Request())
        else:
            flow = InstalledAppFlow.from_client_secrets_file(
               'credentials.json', SCOPES)
            creds = flow.run_local_server(port=0)
        # Save the credentials for the next run
        with open('token.json', 'w') as token:
            token.write(creds.to_json())
        creds_as_bytes = creds.to_json().encode('utf-8')
        base64creds = base64.b64encode(creds_as_bytes).decode("utf-8")
        print(f'Base64 encoded token.json: {base64creds}')

    service = build('admin', 'reports_v1', credentials=creds)

    # Call the Admin SDK Reports API
    print('Getting the last 10 login events')
    results = service.activities().list(userKey='all', applicationName='login',
        maxResults=10).execute()
    activities = results.get('items', [])

    if not activities:
        print('No logins found.')
    else:
        print('Logins:')
        for activity in activities:
            print(u'{0}: {1} ({2})'.format(activity['id']['time'],
                activity['actor']['email'], activity['events'][0]['name']))

if __name__ == '__main__':
    main()

To run this script, you'll need:

A graphical user interface (the script opens a browser to complete authorization)
Python 3.6 or greater
A Google account in the domain you want to collect data from with administrator privileges

Follow these steps:

Get the above script.
Save the script to <any_directory>/devo-collectors/gsuite-reports/credentials.
Copy the credentials.json file downloaded previously to <any_directory>/devo-collectors/gsuite-reports/credentials.
Install the Google Auth API library in the python interpreter being used on your computer.
$ pip install --upgrade google-api-python-client google-auth-httplib2 google-auth-oauthlib
Run the command below in the <any_directory>/devo-collectors/gsuite-reports/credentials directory to create the token.json file. A Google consent window will prompt asking for permission scopes to be accepted, follow the instructions in the browser, and allow the application.
$ python quickstart.py
The script will output a line starting with Base64 enconded token.json: Copy the base64 value as this will be required for the collector configuration. This script was originally developed by Google and has been adapted by Devo.

Run the collector

Once the data source is configured, you can either send us the required information if you want us to host and manage the collector for you (Cloud collector), or deploy and host the collector in your own machine using a Docker image (On-premise collector).

G Suite Report lag times

The lag times in this table show how long it can take before data for specific Admin console reports and audit logs are available.

Item name	Report name	Lag time

Item name	Report name	Lag time
Highlights
Gmail	Gmail report	1-3 days
Drive	Drive report	1-3 days
Hangouts	Hangouts report	1-3 days
Google+	Google+ report	1-3 days
Calendar	Calendar report	1-3 days
Document Link Shared Status	Drive report	1-3 days
Security
External Link Shared Files	Drive report	1-3 days
External Link Shared Files	Security report	1-3 days
Less Secure Apps Access	Security report	1-3 days
2-Step Verification Enrollment	2SV report	1-3 days
Aggregate reports
Accounts	Accounts report	1-3 days
Gmail	Gmail report	1-3 days
Drive	Drive report	1-3 days
Google+	Google+ report	1-3 days
Mobile	Mobile report	1-3 days
Apps usage activity
Files added	Drive report	1-3 days
Total Emails	Gmail report	1-3 days
Total Storage Used (MB)	Quota report	1-3 days
Audit
Admin	Admin audit	almost real-time (a couple of minutes)
Login	Login audit	1-2 days
Drive	Drive audit	almost real-time (a couple of minutes)
Calendar	Calendar audit	tens of minutes (can also go up to a couple of hours)
Jamboard	Jamboard audit	1-3 days
Google+	Google+ audit	1-3 days
Chat	Chat audit	1-3 days
Meet	Meet audit	almost real-time (a couple of minutes)
Voice	Voice audit	1-3 days
Mobile devices	Devices audit	up to a few hours
SAML	SAML audit	up to a few hours
LDAP	LDAP audit	1-3 days
Token	Token audit	a couple of hours
Groups	Groups audit	tens of minutes (can also go up to a couple of hours)
User accounts	User accounts audit	tens of minutes
Access Transparency	Access Transparency audit	almost real-time (a couple of minutes)
Email log search	Email audit	1-3 days

Retrieving report or audit log data for older dates or a wide time range might take so long that, by the time results are available, the most recent log data might no longer be fresh. For tools that require real-time monitoring, use a short time range. Many products listed above (such as Gmail and Google Drive) are relevant for G Suite only, and not for other Google services, such as Cloud Identity.

Learn more about limits and quotas and lag times.

Disclaimer

The API limits the number of requests for your APIs Console project. The API project's maximum number of requests per second (project QPS) is 5 QPS and the maximum number of requests per day (project QPD) is 150,000 QPD across the account. If these limits are exceeded, the server returns an HTTP 503 status code.

You might find that your Admin console reports and audit logs don’t show the latest data, because reports don’t reflect real-time data. The lag times in the table below show how long it can take before data for specific Admin console reports and audit logs is available. Some reports might take longer to display updated information.

There's a small chance that reports and audit logs for some events will be delayed beyond the specified times below. In very rare cases, events may not be reported.

Devo v7.10.0