News & Announcements User Community Developer Community

Welcome to the RingCentral Community

Please note the community is currently under maintenance and is read-only.

Search
Make sure to review our Terms of Use and Community Guidelines.
  Please note the community is currently under maintenance and is read-only.
Home » Developers
Batch retrieving message attachments
Tags: rate limits, errors
Jun 16, 2016 at 7:47am   •   10 replies  •  0 likes
Julio Toledo

We are retrieving user message log for 6 month period containing as many as 2,000 records and hundreds of message attachment URIs (mpegs and pdfs).


Our follow-up attempts to retrieve actual message attachments in a loop is causing rate exceed limit errors.


Does the Batch Request GET method support fetching actual binary text files (i.e.in blocks of 50)?


If not, is there another batch operation we can use to overcome rate exceed limit and accelerate the retrieval of large quantities of attachment binaries?


10 Answers
answered on Jun 18, 2016 at 10:27pm  
We're using php.

I created a support case Friday. Will add this post reference to it.

Support case 04731530

 0
answered on Jun 18, 2016 at 9:37pm  
The X-Rate-Limit-* headers are generated for both API calls and API Explorer calls, however, without the proper inspection you will not see them.

While API Explorer API calls will return the X-Rate-Limit-* headers, it seems that the API Explorer UI does not display them which we will look into. You can view the response headers using Google Chrome's Network Inspection of the API Explorer 'account/~/extension/~/message-store' API call as shown in the screen shot below:



Additionally, when I make an API call to 'account/~/extension/~/message-store', I get the following X-Rate-Limit-* headers (lowercased by Ruby) using the Ruby SDK (https://github.com/grokify/ringcentral-sdk-ruby/) using "pp res.headers":

 "x-rate-limit-group"=>"light",
 "x-rate-limit-limit"=>"50",
 "x-rate-limit-remaining"=>"49",
 "x-rate-limit-window"=>"60"

How are you making API calls and to which endpoints where you are not seeing the X-Rate-Limit-* response headers?

If you create a support case, please reference this post.


 0
answered on Jun 17, 2016 at 1:25pm  
Will do.

I am going to guess our API responses may be missing the headers for the same/similar reason that your API Explorer is also missing the response headers :)


 0
answered on Jun 17, 2016 at 1:21pm  
Well...that's a problem, isn't it. :)

Could you do me a favor and submit a Developer Support case (https://developers.ringcentral.com/api/support-cases/create) and request that your account is inspected and resolved as to why your API responses are missing the X-Rate-Limit-* headers?

 0
answered on Jun 17, 2016 at 11:40am  
Right, I am aware of these response headers, I just have never seen them in any of the API responses that we get for our app.

Our RingCentral responses look identical to the API Explorer responses (no headers).

 0
answered on Jun 17, 2016 at 11:37am  
I had not noticed that the API Explorer does not show these new response headers. I have created a ticket to remedy this in the future.

There are four (4) different response headers returned by the RingCentral API:
  • X-Rate-Limit-Group (string)
    Provides you with the human-readable name which you can reference in the Developer Portal's Rate Limit section for your application for a particular API resource's limits

  • X-Rate-Limit-LImit (integer, fixed number)
    Although a little redundant, this is the header which defines the actual count of requests which can be made based upon the X-Rate-Limit-Group during the X-Rate-Limit-Window value (in seconds)

  • X-Rate-Limit-Remaining (integer, decremented count)
    This header is an approximate value of the number of calls you have remaining in the current limit-measuring time window

  • X-Rate-Limit-Window (integer, in seconds)
    This header defines the length of time the maximum number of requests (as defined by the X-Rate-Limit-Limit header indicates) can be executed before your requests will be throttled

 0
answered on Jun 17, 2016 at 10:01am  
Thanks Benjamin,

I do have a related question about rate limits. I am told all responses to all endpoints include headers that contain rate limit metadata. However I have never seen such headers in any of the API responses we are getting, nor are they included in the API Explorer when you test endpoints there.

Where can I find these elusive headers?

 0
answered on Jun 17, 2016 at 9:39am  
Let me know how things go and if you need further assistance.

 0
answered on Jun 16, 2016 at 2:37pm  
Hi Benjamin,

Thanks for your response.

Fortunately not all message records in the message store contain attachments, as a large percentage of them are SMS text messages :)

Actual figures (per user) are approximately 2,000 messages (or about 7 pages of message store records at 300 per page), and of those, roughly 25% (or about 500) of them contain pdf and mpeg attachments. So if my math is correct, it should take about 12-15 minutes to retrieve all 500 attachments.

The use case is for local caching of message store (on device) upon first time login. Following the first mass retrieval each device will continue to update its local message repository differentially. However, thanks to ubiquity, users will sign in and out of multiple devices, and they expect their data to follow them, so message store will most likely need to be re-fetched multiple times (once per device upon first login).

Even if we were to cut off message store caching in half (3 months), that would still mean retrieving about 250 attachments...

We will experiment with your recommendations.

 1
answered on Jun 16, 2016 at 2:08pm  
Batch requests are counted as a single API request regardless of the total number of items requested in the batch.

According to our documentation about batch requests (following sentence is URL), the Message Store should support batch requests do. https://developers.ringcentral.com/api-docs/latest/index.html#!#BatchRequests.html

HOWEVER, I do not see this referenced in the section of Message Store about Attachments: https://developers.ringcentral.com/api-docs/latest/index.html#!#RefGetMessageAttachment

You could try by sending a small batch of four or five items and see if it works, but I think you will receive an HTTP 400 response.

Instead I would recommend one of the following:

A: Create additional admin users (service users essentially) and execute multiple instances of this application in server-side code using each of these users to fetch the data.
B: Submit a Developer Support case to request that your Application's Usage Plan be reviewed for increase considerations.



Knowing you can use batch requests to obtain the list, and to get the individual Message Info items, you can pre-fetch all the data you know you'll need to get the attachment information.

Using your numbers from above (and doing some rough math):

2000 records * 300 attachments per record === 600,000 attachments (roughly)

The Basic Usage Plans permit 40 requests per minute: 40 * 60 (1 hour) * 24 (1 day) * 7 (1 week) === 403,200

This means you should be able to fetch all 600,000+ attachments in under two weeks (if you ALWAYS honor the rate limiting and do not get throttled).

Is that not sufficient (considering you have not attempted to download these attachments for at least the past 6 months)?

 0



A new Community is coming to RingCentral!

Posts are currently read-only as we transition into our new platform.

We thank you for your patience
during this downtime.

Try Workflow Builder

Did you know you can easily automate tasks like responding to SMS, team messages, and more? Plus it's included with RingCentral Video and RingEX plans!

Try RingCentral Workflow Builder

PRODUCTS
RingEX
Message
Video
Phone
OPEN ECOSYSTEM
Developer Platform
APIs
Integrated Apps
App Gallery
Developer support
Games and rewards

RESOURCES
Resource center
Blog
Product Releases
Accessibility
QUICK LINKS
App Download
RingCentral App login
Admin Portal Login
Contact Sales
© 1999-2024 RingCentral, Inc. All rights reserved. Legal Privacy Notice Site Map Contact Us